Under the assumption that observed responses are generated by an unknown deterministic function corrupted by zero-mean noise of fixed variance, the expected squared prediction error at a query point decomposes exactly into three additive terms: the variance of the estimator across training samples, the squared bias of its expected prediction relative to the true function, and an irreducible noise variance that no learner can remove. Model complexity governs a trade-off between the first two terms — richer hypothesis classes lower bias and raise variance, simpler ones do the reverse — while the third term places a hard floor on achievable error. The decomposition belongs to statistical learning theory and supplies the standard lens for analyzing any estimator in supervised regression or classification, including the degradation of local averaging methods in high dimensions.
Under the assumption that observed responses are generated by an unknown deterministic function corrupted by zero-mean noise of fixed variance, the expected squared prediction error at a query point …