Decision trees are high-variance, unstable learners: small perturbations in the training sample can restructure the tree substantially, because greedy splits chosen near the root propagate their effect through every subtree, and the instability worsens as sample size shrinks. Pruning and regularization mitigate but do not eliminate this variance, so the general remedy is bagging — training many models on randomly resampled subsets of the data and aggregating their predictions — which applies to any unstable classifier and buys stability at the cost of interpretability. The concept also covers the intrinsic limitations of trees (piecewise-constant, non-smooth fits and unavoidable repeated subtrees) and the generalization of the 0/1 loss to an asymmetric class-dependent loss matrix that can be folded into impurity criteria such as the Gini index; the topic belongs to supervised learning within machine learning, specifically the bias-variance and ensemble-methods branch of tree-based models.
Decision trees are high-variance, unstable learners: small perturbations in the training sample can restructure the tree substantially, because greedy splits chosen near the root propagate their effe…