噪声敏感指数控制单索引和多索引模型中的大规模统计-计算差距
A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models
AI总结:
本文研究了单索引和多索引模型中统计与计算差距的形成机制,发现噪声敏感指数决定了这些模型中统计与计算差距的存在和大小。
摘要:
理解学习何时在统计上可行却在计算上困难是高维统计学中的核心挑战。本文在单索引和多索引模型的背景下探讨了这一问题,这些模型是广泛研究的基准,用于探测机器学习方法在高维数据中发现特征的能力。我们的主要贡献是证明噪声敏感指数(NSE)——由激活函数决定的简单量——在这些模型的广泛范围内决定了统计与计算差距的存在和大小。我们首先证明,在具有大加性噪声的单索引模型中,计算瓶颈的出现完全由NSE决定。然后展示同一指数控制了大分离多索引模型的专业化过渡中的统计-计算差距,其中单个组件变得可学习。最后,在分层多索引模型中,我们表明NSE决定了不同方向按顺序学习的最优计算速率。总体而言,我们的结果将NSE识别为连接高维学习中噪声鲁棒性、计算难度和特征专业化的统一属性。
英文摘要:
Understanding when learning is statistically possible yet computationally hard is a central challenge in high-dimensional statistics. In this work, we investigate this question in the context of single- and multi-index models, classes of functions widely studied as benchmarks to probe the ability of machine learning methods to discover features in high-dimensional data. Our main contribution is to show that a Noise Sensitivity Exponent (NSE) - a simple quantity determined by the activation function - governs the existence and magnitude of statistical-to-computational gaps within a broad regime of these models. We first establish that, in single-index models with large additive noise, the onset of a computational bottleneck is fully characterized by the NSE. We then demonstrate that the same exponent controls a statistical-computational gap in the specialization transition of large separable multi-index models, where individual components become learnable. Finally, in hierarchical multi-index models, we show that the NSE governs the optimal computational rate in which different directions are sequentially learned. Taken together, our results identify the NSE as a unifying property linking noise robustness, computational hardness, and feature specialization in high-dimensional learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。