Principal component regression replaces the original correlated predictors with the principal component directions of the centered inputs — the eigenvectors of the covariance matrix, equivalently the right singular vectors of the centered data matrix — ordered so that each successive direction carries the maximum remaining variance within the subspace orthogonal to those already extracted. Because these directions are mutually orthogonal, the regression coefficient for each is obtained by an independent univariate regression of the response on that component, and components are added until the residual is acceptably small, with the intercept supplied by the response mean under centering. The method's defining limitation is that the directions are chosen from the inputs alone: a direction of maximal input variance may confound response values that a lower-variance direction would separate cleanly, which is the motivation for supervised direction-finding methods within statistical learning.
Principal component regression replaces the original correlated predictors with the principal component directions of the centered inputs — the eigenvectors of the covariance matrix, equivalently the…