GRRR：解码器LLM后训练中的重塑、旋转与路由几何
GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-training
- CUNY Graduate Center（纽约市立大学研究生中心）
- The City College of New York（纽约市立大学城市学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究通过SVD分解分析LLM后训练权重变化，发现对角分量（重塑奇异值）的移除通常保留大部分收益，表明后训练主要重新配置和扩展预训练路径，而非改变奇异值。
AI中文摘要：
我们研究了后训练如何改变大型语言模型（LLM）相对于其预训练权重的权重。在12个包含监督微调（SFT）和强化学习（RL）的后训练链中，我们将每次权重更新表达在预训练矩阵的奇异值分解（SVD）框架中。这种分解将三个几何上不同的组成部分的变化分离开来：对角值，它重塑奇异值；非对角值，它旋转预训练输入和输出方向之间的耦合；以及零空间值，它在矩阵原始非零SVD核心之外进行路由。在一个数学评估套件上，我们发现移除对角分量通常能保留后训练带来的大部分收益。这些结果表明，后训练的收益主要来自于重新配置和扩展预训练路径，而非显著改变预训练模型的奇异值。
英文摘要：
We study how post-training changes the weights of Large Language Models (LLMs) relative to their pretrained weights. Across 12 post-training chains with supervised fine-tuning (SFT) and reinforcement learning (RL), we express each weight update in the pretrained matrix's singular value decomposition (SVD) frame. This decomposition separates the changes of three geometrically distinct components: diagonal values, which reshapes singular values; off-diagonal values, which rotates the coupling between pretrained input and output directions; and null-space values, which routes outside the matrix's original nonzero SVD core. On a math evaluation suite, we find that removing the diagonal component usually preserves most of the gains from post-training. These results suggest that post-training gains are carried primarily by reconfiguring and extending pretrained pathways rather than by substantially changing singular values of pre-trained models.
