Partial least squares is a derived-directions method for linear regression that constructs each new direction as a weighted combination of the input variables, weighting each input by the univariate projection of the response onto it, so that the direction reflects how much of the response each variable explains on its own. Successive directions are obtained by orthogonalizing the inputs against the directions already extracted and repeating, which makes the derived directions mutually orthogonal and allows each regression coefficient to be obtained by a univariate regression of the response on that direction alone. Because the response participates in forming the directions, partial least squares balances input-space variance against correlation with the response, distinguishing it from principal component regression, which uses input variance only; extracting as many directions as there are inputs recovers the ordinary least-squares fit exactly.
Partial least squares is a derived-directions method for linear regression that constructs each new direction as a weighted combination of the input variables, weighting each input by the univariate …