发表机构
Stanford University（斯坦福大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究将神经网络权重谱重尾涌现建模为右删失击中时间问题，提出维度校正谱隙定律，并通过理论与实验验证，为隐式自正则化提供可复现的定量描述。
AI 中文摘要
神经网络权重矩阵的经验谱密度具有重尾特征，被广泛用作隐式自正则化的诊断指标，但重尾涌现的步骤复杂度仍缺乏深入理解。我们将谱重尾形成建模为右删失击中时间问题：在观测时间范围内未达到重尾诊断指标的运行被视为删失而非丢弃。在受控的全批量教师-学生动力学中，我们发现仅靠第一步的尖峰-体间隙无法解释起始时间。相反，有限起始回归支持维度校正谱隙定律，(\tau_{\mathrm{HT}}\approx C\Delta_1^{-\gamma}d^\rho)，在330次完整运行中，(R^2=0.683)，(\gamma=0.626)，(\rho=0.772)。右删失对数正态加速失效时间模型进一步支持维度校正模型优于仅间隙模型，将AIC从706.62改善至628.70。理论上，我们证明了线性网络中精确的早期损失动力学不能决定因子谱尾部，仅Adam递推本身不意味着谱重分布，且投影奇异基扩展意味着谱尾势的收缩，从而得出维度校正击中时间界。实证上，投影核轮廓支持充分的扩展机制，Adam和AdamW在测试网格下表现一致，GD和signGD在相同机制下未达到起始点，真实预训练的Qwen2.5-0.5B和Pythia-70M transformer权重相对于匹配的高斯零模型显示出非高斯谱尾结构。结果是一个可复现的谱击中时间定律，具有严格的条件理论，而非声称Adam必然从第一性原理产生重尾。
英文摘要
Heavy-tailed empirical spectral densities of neural-network weight matrices are widely used as diagnostics of implicit self-regularization, but the step complexity of heavy-tail emergence remains poorly understood. We formulate spectral heavy-tail formation as a right-censored hitting-time problem: a run that does not reach a heavy-tail diagnostic within the observation horizon is treated as censored rather than discarded. In controlled full-batch teacher--student dynamics, we find that the first-step spike--bulk gap alone does not explain onset time. Instead, finite-onset regression supports a dimension-corrected spectral-gap law, (τ_{\mathrm{HT}}\approx CΔ_1^{-γ}d^ρ), with (R^2=0.683), (γ=0.626), and (ρ=0.772) across 330 completed runs. Right-censored lognormal accelerated-failure-time models further favor the dimension-corrected model over a gap-only model, improving AIC from 706.62 to 628.70. Theoretically, we prove that exact early loss dynamics in linear networks do not determine factor spectral tails, that Adam recurrences alone do not imply spectral redistribution, and that projected singular-basis spreading implies contraction of a spectral-tail potential and hence a dimension-corrected hitting-time bound. Empirically, projected-kernel profiles support the sufficient spreading mechanism, Adam and AdamW agree under tested grids, GD and signGD do not reach onset in the same regimes, and real pretrained Qwen2.5-0.5B and Pythia-70M transformer weights show non-Gaussian spectral-tail structure relative to matched Gaussian nulls. The result is a reproducible spectral hitting-time law with rigorous conditional theory, not a claim that Adam necessarily generates heavy tails from first principles.
