A loss function is a differentiable scalar-valued function that maps a model's parameters and a batch of labeled examples to a single number measuring how badly the model's predictions disagree with the targets. The core mechanism is that prediction error is aggregated into one objective (squared error for continuous targets, cross-entropy for categorical ones) so that model quality becomes a value to be minimized over the parameter space rather than an informal judgment. Because the objective is chosen to be differentiable in the parameters, it defines the surface whose slope any first-order optimization method reads, which is what makes 'improving the model' a well-posed minimization problem in supervised learning.
In machine learning theory, a loss function serves as a differentiable objective metric that quantifies the discrepancy between predicted outputs and ground truth labels to guide model optimization. …