A decision tree is a supervised learning model that classifies by querying one feature at a time against a threshold, thereby recursively partitioning the input space into axis-aligned rectangular regions, each region receiving a single predicted label. The model's inductive bias is the assumption that the target function is well approximated by such recursively generated rectangles — the analogue of the linearity assumption made by linear classifiers — and this recursive construction, rather than an arbitrary tiling of the space, is what makes the partition compactly expressible as a tree and computationally tractable. Decision trees are nonparametric universal approximators whose parameter count grows without bound as the tree deepens, and their defining practical virtue is interpretability, standing in direct contrast to black-box models such as neural networks.
A decision tree is a supervised learning model that classifies by querying one feature at a time against a threshold, thereby recursively partitioning the input space into axis-aligned rectangular re…