Positive Semi-Definiteness and the Covariance Matrix

why variance along every direction is non-negative, which is what makes PCA's ordering of components meaningful

Default Guide Video

Proof That a Covariance Matrix Is Positive Semi-Definite

A covariance matrix of any random vector is positive semi-definite: for a random vector X with density rho(X) and mean vector mu, the covariance matrix Sigma is defined as the expectation of (X − mu)…