随机特征方法和神经网络的平滑偏差原则

中文摘要 AI 辅助

我们在经典的非参数回归设置中研究谱正则化方法的数据驱动早停。基于偏差原则，我们提出了一种适用于一般核估计器的多尺度停止规则，并表明与以往方法不同，它在设定正确的情况下对所有平滑度水平实现了完全自适应性。我们工作的一个关键贡献是基于随机特征近似的扩展，这在大数据集上降低了计算成本，同时保持了极小极大最优的统计保证。我们的程序不仅选择最优停止时间，还提供了完全数据驱动的随机特征数量选择，以达到最优速率。通过随机特征与神经正切核机制下神经网络之间已建立的联系，我们的方法进一步为网络宽度提供了原则性的、数据驱动的建议。我们证明了由此同时选择的宽度和停止时间使神经网络能够在没有平滑度或容量参数先验知识的情况下达到极小极大最优学习速率。

英文摘要

We study data-driven early stopping for spectral regularisation methods in the classical non-parametric regression setting. Building on the discrepancy principle, we propose a multi-scale stopping rule that applies to general kernel estimators and show that, unlike previous approaches, it achieves full adaptivity over all smoothness levels in the well-specified case. A key contribution of our work is an extension based on random feature approximations, which reduces computational cost on large datasets while preserving minimax-optimal statistical guarantees. Our procedure not only selects an optimal stopping time but also provides a fully data-driven choice of the number of random features needed to achieve optimal rates. Through the established connection between random features and neural networks in the neural tangent kernel regime, our method further yields a principled, data-driven recommendation for the network width. We prove that the resulting simultaneously chosen width and stopping time allow neural networks to attain minimax-optimal learning rates without prior knowledge of smoothness or capacity parameters.

发表机构

• Technical University of Braunschweig（布伦瑞克工业大学）

机构由 AI 辅助整理，请以论文原文为准。