When training data are not linearly separable the hard-margin support vector machine has no feasible solution, so the margin constraint is relaxed by introducing a non-negative slack variable ξᵢ for each point, replacing yᵢf(xᵢ) ≥ M with yᵢf(xᵢ) ≥ M(1 − ξᵢ); ξᵢ measures the *relative* fraction of the margin by which point i is violated, with ξᵢ = 0 for points correctly placed outside the margin and ξᵢ > 1 for misclassified points. The total slack is bounded by a budget, which — exactly as the ridge and LASSO constraints are exchanged for a penalty term — is equivalently absorbed into the objective as a multiplier C, producing the soft-margin formulation whose trade-off is that large C penalizes violations heavily and yields a narrow margin fitting the training data closely, while small C buys a wider, more robust margin at the cost of training errors. In the dual, slack changes the constraint on the multipliers from αᵢ ≥ 0 to 0 ≤ αᵢ ≤ C, and the support vectors now include every point on or inside the margin as well as every misclassified point; the topic belongs to supervised classification within machine learning, specifically the constrained-optimization formulation of large-margin classifiers.
When training data are not linearly separable the hard-margin support vector machine has no feasible solution, so the margin constraint is relaxed by introducing a non-negative slack variable ξᵢ for …