放大相邻令牌差异：利用Shift-FFN增强长链式思维推理
Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN
AI总结：
针对长链式思维推理中全参数或低秩LoRA微调导致的循环推理问题，本文提出Shift-FFN，通过编辑当前令牌表征放大相邻差异，实验证明其结合LoRA在数学推理上更优。
摘要：
近期，诸如OpenAI-o1和DeepSeek-R1等模型通过长链式思维（Long-CoT）推理在复杂推理任务上展示了卓越的性能。尽管将这种能力蒸馏到学生模型中可以显著提升其性能，但本文发现，在长CoT数据上使用全参数微调或低秩LoRA微调LLM，往往会导致循环推理（Cyclical Reasoning），即模型反复重复之前的推理步骤直到达到最大长度限制。进一步分析揭示，相邻令牌之间的表征差异较小与循环推理倾向较高相关。为缓解此问题，本文提出Shift前馈网络（Shift-FFN），一种新颖的方法，在将当前令牌的表征输入FFN之前，用前一个令牌的表征对其进行编辑。该架构动态地放大相邻令牌之间的表征差异。在多个数学推理任务上的大量实验表明，与全参数微调和标准LoRA相比，结合Shift-FFN的LoRA在各种数据规模下均能实现更高的准确率和更低的循环推理率。我们的数据和代码可在https://anonymous.4open.science/r/Shift-FFN获取。
英文摘要：
Recently, models such as OpenAI-o1 and DeepSeek-R1 have demonstrated remarkable performance on complex reasoning tasks through Long Chain-of-Thought (Long-CoT) reasoning. Although distilling this capability into student models significantly enhances their performance, this paper finds that fine-tuning LLMs with full parameters or LoRA with a low rank on long CoT data often leads to Cyclical Reasoning, where models repeatedly reiterate previous inference steps until the maximum length limit. Further analysis reveals that smaller differences in representations between adjacent tokens correlates with a higher tendency toward Cyclical Reasoning. To mitigate this issue, this paper proposes Shift Feedforward Networks (Shift-FFN), a novel approach that edits the current token's representation with the previous one before inputting it to FFN. This architecture dynamically amplifies the representation differences between adjacent tokens. Extensive experiments on multiple mathematical reasoning tasks demonstrate that LoRA combined with Shift-FFN achieves higher accuracy and a lower rate of Cyclical Reasoning across various data sizes compared to full fine-tuning and standard LoRA. Our data and code are available at https://anonymous.4open.science/r/Shift-FFN
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
