漂移契约：面向深度鲁棒局部学习的谱更新
The Drift Contract: Spectral Updates for Depth-Robust Local Learning
AI总结：
本研究将Muon谱更新几何应用于局部学习，提出漂移契约步长规则，显著提升深度鲁棒性并简化超参数调整，在CIFAR-10上优于局部Adam，且证明优势源于谱几何本身。
AI中文摘要：
局部学习使用每个层自身的辅助损失进行训练，且不进行全局反向传播，这使得层更新在结构上具有并行性。两个问题一直使其处于边缘地位：随着深度增加，精度下降；超参数脆弱。我们将Muon风格的谱更新几何（动量正交化与谱步长缩放）应用于逐层局部更新，这是此前未被研究的交叉点。在CIFAR-10 MLP基准测试中，使用局部线性头，从宽度128到2048、深度12到48的测试网格中，单一步长设置即为最优值，而局部Adam需要在两个轴上重新调整，且在深度48时仍会崩溃（按深度重新调整后为31.3%，使用其深度12设置迁移时为19%，而谱更新在不变设置下为42.7%）。在五个种子和宽度512下，谱更新明显领先于局部Adam（48.9±0.5对46.6±0.3）。前瞻性指定的控制实验表明，迁移性和大部分深度鲁棒性归因于谱几何本身，而非其上的任何步长规则。我们进一步将步长表述为漂移契约，lr = epsilon / RMS(input)，该契约限制了每层权重引起的预激活变化，以其当前输入为条件。该契约在测量基线的最佳固定学习率基础上带来小幅提升，使步长可解释，并提供标准优化器无法提供的逐层、输入条件的漂移界限。我们报告一个负面结果：在主干中使用RMSNorm和权重衰减时，谱更新的稳定性优势归于全局训练而非局部训练，因此局部优势恰好集中在无归一化的地方。
英文摘要：
Local learning trains each layer with its own auxiliary loss and no global backward pass, which makes layer updates structurally parallel. Two problems have kept it marginal: accuracy degrades as depth grows, and hyperparameters are fragile. We apply Muon-style spectral update geometry (momentum orthogonalization with spectral step scaling) to per-layer local updates, an intersection not previously studied. On CIFAR-10 MLP benchmarks with local linear heads, a single step-size setting is the best value in our tested grids from width 128 to 2048 and from depth 12 to 48, while local Adam requires re-tuning along both axes and still collapses at depth 48 (31.3 percent re-tuned per depth, 19 percent with its depth-12 setting transferred, vs 42.7 percent for the spectral update at its unchanged setting). At five seeds and width 512 the spectral update leads local Adam by a clear margin (48.9 +/- 0.5 vs 46.6 +/- 0.3). Prospectively specified controls attribute the transfer and most of the depth robustness to the spectral geometry itself rather than to any step-size rule on top of it. We additionally formulate the step size as a drift contract, lr = epsilon / RMS(input), which bounds each layer's weight-induced pre-activation change per step, conditioned on its current input. The contract yields a small gain over the best fixed learning rate where that baseline is measured, makes the step size interpretable, and provides a per-layer, input-conditioned drift bound that standard optimizers do not offer. We report one negative result: with RMSNorm and weight decay in the trunk, the stability benefit of spectral updates accrues to global rather than local training, so the local advantage concentrates precisely where normalization is absent.
