gradient descent

Syllable breakdown

Dictionary meaning

an iterative optimisation algorithm that minimises a loss function by repeatedly adjusting model parameters in the direction opposite to the gradient of the loss with respect to those parameters; the step size is controlled by the learning rate hyperparameter; variants including stochastic gradient descent, mini-batch gradient descent, and adaptive methods such as Adam and RMSProp are the workhorses of training neural networks and many other machine learning models