Naive Bayes is a generative classifier that obtains the posterior P(g | x) from Bayes' rule as the product of the class prior P(g) and the class-conditional density P(x | g) (the likelihood), normalized by the marginal P(x) obtained by summing the numerator over all classes. Because the class-conditional joint density cannot be estimated directly in high-dimensional spaces owing to data sparsity, the Naive Bayes assumption is imposed: given the class label, the features are mutually independent, so the joint class-conditional factorizes into a product of conditional marginals, each estimable by simple counting (for discrete features) or by a parametric form such as a Gaussian (for continuous ones). The assumption is severe and frequently false in the semantic sense, yet the classifier performs competitively — notably in very high-dimensional settings where distance-based methods degrade — and its remaining practical concerns are the unimodality of a Gaussian marginal (addressed by binning or a mixture model) and the zero-probability problem for unseen feature values (addressed by smoothing, which must be applied cautiously when the unobserved value space is large).
Naive Bayes is a generative classifier that obtains the posterior P(g | x) from Bayes' rule as the product of the class prior P(g) and the class-conditional density P(x | g) (the likelihood), normali…