Because both the dual of the maximum-margin optimization problem and the resulting decision function depend on the training data only through inner products between pairs of points, any function that returns the inner product of two points under a basis expansion can be substituted directly for that inner product — this substitution is the kernel trick. A kernel is a symmetric, positive semi-definite similarity function defined on the original input space whose value equals an inner product in a (possibly infinite-dimensional) feature space, so the expanded representation never has to be constructed or stored. The concept belongs to statistical learning theory and kernel methods, and it generalizes support vector machines beyond real-valued vector data to any domain for which a valid kernel can be defined.
Because both the dual of the maximum-margin optimization problem and the resulting decision function depend on the training data only through inner products between pairs of points, any function that…