MIT OpenCourseWare
48 minutes
Projection Matrix Formulas in Linear Algebra
A projection matrix drops a vector straight down onto a subspace, such as a line or a plane. If the columns of $A$ are linearly independent, the matrix that projects onto the column space of $A$ is $P = A(A^T A)^{-1} A^T$. Every projection matrix is symmetric ($P^T = P$) and idempotent ($P^2 = P$), which means projecting twice is the same as projecting once. Projections solve least-squares problems. When $A\mathbf{x} = \mathbf{b}$ has no exact solution, the best choice $\hat{\mathbf{x}}$ makes the error $\mathbf{e} = \mathbf{b} - A\hat{\mathbf{x}}$ perpendicular to the column space of $A$. That condition gives the normal equations $A^T A \hat{\mathbf{x}} = A^T \mathbf{b}$.