合并阶梯性质：两层神经网络SGD学习稀疏函数的必要且几乎充分的条件
The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks
AI总结：
研究平均场机制下两层神经网络通过SGD学习二元输入上潜在低维稀疏函数的可学习性，提出“合并阶梯性质”作为必要且几乎充分条件，并证明非线性训练的必要性。
摘要：
目前已知如何刻画神经网络在两种极端参数化下能通过SGD学习的函数：线性机制下的神经网络，以及无结构约束的神经网络。然而，对于主要关注的参数化（非线性但常规的网络），尽管已有重大进展，但尚未实现紧密的刻画。我们在此方向上迈出了一步，考虑在平均场机制下通过SGD训练的深度为2的神经网络。我们考虑二元输入上依赖于潜在低维子空间（即少量坐标）的函数。这一机制备受关注，因为人们尚不清楚神经网络如何常规地处理高维数据集并适应潜在的低维结构而不受维度灾难的影响。因此，我们在大环境维度d下研究具有O(d)样本复杂度的SGD可学习性。我们的主要结果刻画了一种层次化性质，即“合并阶梯性质”，它在该设定下既是必要的，也几乎是充分的。我们进一步表明非线性训练是必要的：对于这类函数，任何特征映射（例如NTK）上的线性方法都无法高效学习。关键工具包括一项适用于定义在低维潜在空间上函数的新的“无维度”动力学近似结果、一个基于多项式恒等测试的全局收敛证明，以及对非几乎正交函数线性方法下界的改进。
英文摘要：
It is currently known how to characterize functions that neural networks can learn with SGD for two extremal parameterizations: neural networks in the linear regime, and neural networks with no structural constraints. However, for the main parametrization of interest (non-linear but regular networks) no tight characterization has yet been achieved, despite significant developments. We take a step in this direction by considering depth-2 neural networks trained by SGD in the mean-field regime. We consider functions on binary inputs that depend on a latent low-dimensional subspace (i.e., small number of coordinates). This regime is of interest since it is poorly understood how neural networks routinely tackle high-dimensional datasets and adapt to latent low-dimensional structure without suffering from the curse of dimensionality. Accordingly, we study SGD-learnability with $O(d)$ sample complexity in a large ambient dimension $d$. Our main results characterize a hierarchical property, the "merged-staircase property", that is both necessary and nearly sufficient for learning in this setting. We further show that non-linear training is necessary: for this class of functions, linear methods on any feature map (e.g., the NTK) are not capable of learning efficiently. The key tools are a new "dimension-free" dynamics approximation result that applies to functions defined on a latent space of low-dimension, a proof of global convergence based on polynomial identity testing, and an improvement of lower bounds against linear methods for non-almost orthogonal functions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
