When a decision tree is grown for classification rather than regression, squared error is replaced by a measure of node impurity computed from the estimated class proportions p̂_mk (the fraction of points of class k in region m), and the split is chosen to minimize the weighted sum of that measure over the child regions. Three measures are standard: misclassification error, defined as 1 − p̂_m,k(m) where k(m) is the argmax class assigned to the whole region; the Gini index, adapted from the economists' measure of wealth disparity to reward class distributions that are skewed rather than uniform; and cross entropy or deviance, the Shannon-entropy form whose reduction under a split equals the information gain, so that minimizing cross entropy and maximizing information gain are equivalent. Because Gini and entropy are relative measures useful for comparing candidate splits while misclassification error is the quantity ultimately evaluated, the tree may be grown with any of them but should be pruned under cost-complexity pruning using misclassification error.
When a decision tree is grown for classification rather than regression, squared error is replaced by a measure of node impurity computed from the estimated class proportions p̂_mk (the fraction of po…