scaling laws

Definition of scaling laws

1. : observed patterns showing that model performance improves in a predictable way as you increase model size, training data, and compute

   • Scaling laws gave labs the confidence to spend billions on bigger training runs.
   • Some argue the old scaling laws are slowing down; others say they've just moved to inference.