Bias-Variance Tradeoff
Expected test error decomposes into bias (error from a model too simple to represent the truth), variance (sensitivity to the particular training sample), and irreducible noise. Increasing model capacity trades bias for variance; regularization, more data, and ensembling move along the curve — diagnose which side you are on by comparing training and validation error.
This Concept is waiting for its first lesson!
Expected test error decomposes into bias (error from a model too simple to represent the truth), variance (sensitivity to the particular training sample), and irreducible noise. Increasing model capacity trades bias for variance; regularization, more data, and ensembling move along the curve — diagnose which side you are on by comparing training and validation error.
Are you a teacher? Sign in to start contributing.
Sign In