Logits Replay + MoClip：稳定、低成本的微调后训练，遗忘最小化
Logits Replay + MoClip: Stabilized, Low-Cost Post-Training with Minimal Forgetting
AI总结：
提出Logits Replay + MoClip两阶段框架，通过压缩logits监督和稳定优化，在提升领域性能的同时缓解遗忘，降低训练成本超40%。
摘要：
大型语言模型（LLMs）在后训练中常常面临权衡：在特定领域的改进往往以牺牲通用能力为代价。现有解决方案试图通过正则化、选择性参数更新或基于数据的重放来缓解这一矛盾，但每种方法都在计算、数据访问或适应性方面带来显著成本。近期研究表明，训练信号可以压缩为logits的子集而不会造成严重的精度损失，这为高效适应指明了一条路径。然而，朴素的截断会破坏优化稳定性并加剧遗忘。我们提出了Logits Replay + MoClip，一个两阶段框架，在logit空间中压缩监督信号，并在更新层面稳定优化。在阶段0中，我们记录覆盖概率阈值的动态Top-K token子集，始终包含金标标签。在阶段1中，我们重放这些紧凑子集以计算精确的重新归一化损失，避免完整的softmax计算并隐式正则化。为确保稳定性，我们设计了MoClip优化器，它限制梯度动量旋转，并应用基于arctan2的更新重新缩放。实验上，我们的方法在通信技术（CT）和NL2SQL任务上提升了领域性能，同时在通用基准（MMLU、BBH、GPQA、MATH）上缓解了遗忘，并将训练成本降低了40%以上。这些贡献共同提供了一条可扩展、架构无关的路径，用于LLMs的领域适应，而不牺牲泛化能力。
英文摘要：
Large language models (LLMs) often face a trade-off in post-training: improvements on specialized domains frequently come at the expense of general capabilities. Existing solutions attempt to mitigate this tension via regularization, selective parameter updates, or data-centric replay, but each imposes significant costs in computation, data access, or adaptability. Recent work has shown that training signals can be compressed to subsets of logits without severe accuracy loss, suggesting a path toward efficient adaptation. However, naive truncation destabilizes optimization and exacerbates forgetting. We introduce Logits Replay + MoClip, a two-stage framework that compresses supervision in the logit space and stabilizes optimization at the update level. In Stage 0, we record dynamic Top-K token subsets that cover a probability threshold, always including the gold label. In Stage 1, we replay these compact subsets to compute exact renormalized losses, avoiding full softmax computation and implicitly regularizing. To ensure stability, we design MoClip, an optimizer that caps gradient-momentum rotation and applies an arctan2-based rescaling of updates. Empirically, our method improves domain performance on Communication Technology (CT) and NL2SQL tasks while mitigating forgetting on general benchmarks (MMLU, BBH, GPQA, MATH), and reduces training cost by over 40%. Together, these contributions offer a scalable, architecture-agnostic path for domain adaptation of LLMs without sacrificing generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
