AI 中文总结
研究神经网络宽度缩放问题，引入有效对齐维度，推导训练与测试梯度内积的均值和方差以获失准概率上界，集成到框架得测试风险改进条件，实验表明宽模型有效对齐维度大、失准低，对齐统计量可预测损失变化。
AI 中文摘要
现有的神经网络宽度理论描述的是渐近极限，但对于从有限训练数据中确定的扩展方向在未见数据上是否仍然有益，提供的指导有限。我们针对保持函数的残差扩展研究了这个问题，并引入了有效对齐维度，这是一个描述激活梯度信号噪声几何结构的可测量量。通过推导独立估计的训练和测试梯度之间内积的精确均值和方差，我们得到了失准概率的有限样本上界。该上界仅取决于有效对齐维度和有效样本大小，需要有限的二阶矩和非零总体梯度，无需协方差谱假设或规定的宽度增长率。我们将此证书集成到训练-测试残差扩展框架中，得到了测试风险改进的高概率条件。在宽度可控的LLaMA风格Transformer、Pythia和ResNet-20上的实验表明，更宽的模型表现出更大的有效对齐维度和更低的经验失准。直接残差干预证实，对齐统计量可以预测留出损失变化的符号和大小。
英文摘要
Existing theories of neural-network width characterize asymptotic limits, but provide limited guidance on whether an expansion direction identified from finite training data remains beneficial on unseen data. We study this problem for function-preserving residual expansion and introduce the effective alignment dimension, a measurable quantity describing the signal-noise geometry of activation gradients. By deriving the exact mean and variance of the inner product between independently estimated training and test gradients, we obtain a finite-sample upper bound on misalignment probability. The bound depends only on the effective alignment dimension and an effective sample size, requiring finite second moments and a nonzero population gradient, without covariance spectral assumptions or prescribed width-growth rates. We integrate this certificate into the train-test residual-expansion framework, yielding a high-probability condition for test-risk improvement. Experiments across width-controlled LLaMA-style Transformers, Pythia, and ResNet-20 show that wider models exhibit larger effective alignment dimensions and lower empirical misalignment. Direct residual interventions confirm that the alignment statistic predicts the sign and magnitude of held-out loss changes.
Comments28 pages; preprint
