gradient descent

Definition of gradient descent

1. : an optimization method that improves a model step by step by moving its parameters a little in whichever direction reduces the error the most

   • Picture gradient descent as walking downhill in fog, one careful step at a time.
   • The learning rate controls how big each step of gradient descent is.