Cartis, Hauser, Liu, Welzel and Zhu's efficient implementation and numerical study of high-order adaptive-regularization tensor methods (ARp), focused on the third-order method AR3, for unconstrained nonconvex optimization. Contributions include extending the interpolation-based regularization-parameter update from p=2 to p>=3, characterizing how the local minima of the regularized subproblem differ between p=2 and p>=3, a pre-rejection technique that discards transient subproblem minimizers before any function evaluation, studies of subproblem termination and initial regularization, benchmarks showing tuned AR3 variants can beat second-order AR2 in objective/derivative/subproblem-solve counts, and a modular MATLAB package of AR2/AR3 variants (including Hessian- and tensor-free ones).
This paper develops and studies efficient implementations of high-order adaptive-regularization tensor methods (ARp) for unconstrained nonconvex optimization, with a focus on the third-order method A…