Expectation Maximization is an iterative method for maximum-likelihood estimation when the data are incomplete: the observed data X are augmented by hidden data Z, and although the marginal likelihood of X alone places a summation inside the logarithm and is therefore intractable, the complete-data joint likelihood of (X, Z) belongs to a parameterized family whose likelihood is easy to compute. Each iteration performs an E step, computing the posterior distribution of Z under the current parameter guess and forming the Q function — the expected complete-data log likelihood under that posterior — and an M step, maximizing Q to obtain the next parameter estimate; because the posterior responsibilities enter Q as constants, the summations move outside the logarithm and the maximization reduces to the closed-form single-component estimation problem. EM belongs to statistical estimation within machine learning and is the general machinery behind latent-variable models: applied to a Gaussian mixture it reproduces the mixture update equations exactly, and in the limiting case of spherical components with vanishing shared variance and equal mixing weights it degenerates to the k-means algorithm.
Expectation Maximization is an iterative method for maximum-likelihood estimation when the data are incomplete: the observed data X are augmented by hidden data Z, and although the marginal likelihoo…