上下文拖累：上下文中的错误如何影响大语言模型的推理
Contextual Drag: How Errors in the Context Affect LLM Reasoning
AI总结：
研究发现上下文中的错误会偏向下文生成的推理错误，导致性能下降，并指出缓解策略无法完全消除该问题。
摘要：
许多大语言模型（LLM）的自我改进流程都基于一个假设：模型可以通过反思过去的错误来改进。我们研究了一种现象，称为上下文拖累：上下文中存在失败尝试会偏向下一次生成偏向于结构相似的错误。在对11个专有和开源模型在8个推理任务上的评估中，上下文拖累导致性能下降10-20%，且在模型中存在严重上下文拖累的情况下，迭代自我改进可能演变为自我恶化。通过使用树编辑距离进行的结构分析发现，后续推理轨迹继承了来自上下文的结构相似的错误模式。我们证明，外部反馈或成功的自我验证不足以消除这种影响。尽管通过如回退行为微调和上下文去噪等缓解策略可以带来部分改进，但它们无法完全恢复基线性能，将上下文拖累定位为当前推理架构中的持久性失败模式。
英文摘要：
Central to many self-improvement pipelines for large language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed contextual drag: the presence of failed attempts in the context biases subsequent generations toward structurally similar errors. Across evaluations of 11 proprietary and open-weight models on 8 reasoning tasks, contextual drag induces 10-20% performance drops, and iterative self-refinement in models with severe contextual drag can collapse into self-deterioration. Structural analysis using tree edit distance reveals that subsequent reasoning trajectories inherit structurally similar error patterns from the context. We demonstrate that neither external feedback nor successful self-verification suffices to eliminate this effect. While mitigation strategies such as fallback-behavior fine-tuning and context denoising yield partial improvements, they fail to fully restore baseline performance, positioning contextual drag as a persistent failure mode in current reasoning architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
