预训练数据规模会反转Muon-AdamW的微调差距
Pretraining Data Scale Reverses the Muon-AdamW Fine-Tuning Gap
AI总结：
该研究发现预训练数据规模足够大时，微调后AdamW性能会超过Muon，引入边际-扩散比解释该反转，线性联想记忆中梯度下降的该比值随数据预算增长会超过Muon。
摘要：
Muon近期成为预训练大语言模型的有前景优化器，在最小化预训练损失方面比AdamW更快。但目前尚不清楚Muon的优势在微调后是否依然存在。本文研究了Muon和AdamW的下游行为，发现了一个令人惊讶的趋势：当预训练数据预算足够大时，微调后AdamW的性能会超过Muon。此外，随着模型规模增大，AdamW超过Muon所需的预训练token-参数比例会逐渐变小。为解释这种反转，我们将微调视为对输出的扰动，引入了“边际-扩散比”（margin-to-spread ratio），该指标衡量正确logit相对于错误logit的扩散程度高出多少。在语言模型中，Muon在小数据预算下具有更大的边际-扩散比，而AdamW在大数据预算下具有更大的该比值，这与微调趋势一致。在线性联想记忆中，我们证明随着数据预算增长，梯度下降的边际-扩散比在一次更新后会超过Muon的该比值，且模拟显示该趋势在长时间训练后依然存在。
英文摘要：
Muon has recently emerged as a promising optimizer for pretraining large language models, offering a speedup over AdamW in minimizing pretraining loss. However, it remains unclear whether Muon's advantage persists after fine-tuning. In this paper, we study the downstream behavior of Muon and AdamW and uncover a surprising trend: beyond a sufficiently large pretraining data budget, AdamW outperforms Muon after fine-tuning. Moreover, as model size increases, AdamW overtakes Muon at progressively smaller pretraining token-parameter ratios. To explain this reversal, we view fine-tuning as a perturbation of the outputs and introduce the margin-to-spread ratio, which measures how far the correct logit lies above the wrong ones relative to how widely the wrong logits are spread. In language models, Muon has the larger margin-to-spread ratio at small data budgets and AdamW at large ones, mirroring the fine-tuning trend. In a linear associative memory, we prove that after one update the margin-to-spread ratio of gradient descent gains on that of Muon as the data budget grows, and simulations show that this trend persists after extended training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
