Deep P-Spline (DPS) is a statistical-learning method whose novel contribution is identifying neuron selection in a deep neural network with knot placement in a penalized-spline (P-spline) basis expansion, so that a single difference penalty automates the otherwise ad hoc choice of network structure. This yields a latent-variable model fit by an Expectation-Conditional-Maximization (ECM) algorithm with theoretical guarantees for structure tuning, and, read as nonparametric regression, is proven to overcome the curse of dimensionality that cripples classical smoothers on high-dimensional inputs. DPS thereby unifies deep regression with penalized spline smoothing and is demonstrated on computer-experiment surrogate modeling and image-data regression.
This statistics / statistical-machine-learning paper introduces Deep P-Spline (DPS), a method that recasts the hard problem of selecting a deep neural network's structure as the classical problem of …