Browse Teach Curate

Gradient Descent

Iteratively updates parameters by stepping opposite the loss gradient, with a learning rate controlling step size. Stochastic minibatch variants trade exact gradients for far cheaper, noisier steps.

jeremy Video

Gradient Descent Minimizing Neural Network Cost Function via Backpropagation

Gradient descent serves as a first-order optimization algorithm designed to minimize differentiable cost functions by iteratively adjusting model parameters in the direction opposite to the gradient …