This lecture develops the mean-square convergence analysis of the Least Mean Squares (LMS) adaptive filtering algorithm, showing that the steady-state mean-square error exceeds the minimum attainable error by an excess term expressed as a weighted sum of the diagonal entries of a transformed weight-error covariance matrix. The analysis belongs to adaptive filter theory / stochastic signal processing, and relies on eigendecomposition of the input autocorrelation matrix (via a unitary similarity transform) to diagonalize the weight-error covariance recursion, combined with statistical-independence and joint-Gaussian assumptions on the input, desired response, and weight-error vectors. It relates to the parent discipline of estimation theory by extending deterministic linear-transformation properties of mean and covariance vectors, and properties of jointly Gaussian random vectors under linear transformation, to derive a recursive update for the transformed weight-error covariance matrix that governs the algorithm's excess mean-square error behavior.
This lecture develops the mean-square convergence analysis of the Least Mean Squares (LMS) adaptive filtering algorithm, showing that the steady-state mean-square error exceeds the minimum attainable…