大型语言模型顺序后训练中的表示坍缩
Representation Collapse in Sequential Post-Training of Large Language Models
AI总结：
研究大型语言模型在顺序后训练阶段中内部表示逐渐压缩为低秩、各向异性且同质的特征空间，并提出轻量级干预措施以保持未来可学习性。
摘要：
大型语言模型现在通过一系列后训练阶段进行适配，而不是通过单次指令微调。本文研究这种顺序后训练是否逐渐将内部表示压缩为低秩、各向异性且同质的特征空间。我们定义了一套针对隐藏状态、logits、token轨迹和LoRA更新的测量方法，并利用它来分析在受控阶段顺序下的监督微调、偏好优化、安全/拒绝调优、数学和代码专业化以及长思维链调优。中心假设是，过度的表示集中不仅仅是几何上的奇特性：它预示着后期适配中可塑性降低、域外泛化能力减弱以及校准效果变差。我们进一步评估了轻量级干预措施，包括混合域重放、特征刷新、表示多样性正则化和LoRA更新去相关，作为在不放弃后训练行为收益的情况下保持未来可学习性的方法。
英文摘要：
Large language models are now adapted through chains of post-training stages rather than through a single instruction-tuning pass. This paper studies whether such sequential post-training gradually compresses internal representations into low-rank, anisotropic, and homogeneous feature spaces. We define a measurement suite for hidden states, logits, token trajectories, and LoRA updates, and we use it to analyze supervised fine-tuning, preference optimization, safety/refusal tuning, math and code specialization, and long chain-of-thought tuning under controlled stage orderings. The central hypothesis is that excessive representation concentration is not merely a geometric curiosity: it predicts reduced plasticity during later adaptation, weaker out-of-domain generalization, and poorer calibration. We further evaluate lightweight interventions, including mixed-domain replay, feature refresh, representation diversity regularization, and LoRA update decorrelation, as ways to preserve future learnability without giving up the behavioral gains of post-training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
