arXiv 2503.111542025-03-14cs.CLcs.AI

不要断章取义：用于增强大语言模型思维链推理的注意力干预

Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models

Alibaba Cloud Computing（阿里巴巴云计算）；College of Software, Zhejiang University（浙江大学软件学院）；Zhejiang University of Technology（浙江工业大学）

Shaotian Yan, Chen Shen, Wenxiao Wang, Liang Xie, Junjie Liu, Jieping Ye

备注：: Accepted by ICLR2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究少样本 CoT 示范中局部 token 干扰 LLM 推理的问题，通过动态追踪注意力机制提出 FAI，识别并调整误导性 token 的注意力权重，在多个基准上取得稳定提升。

摘要：

少样本思维链（Chain-of-Thought, CoT）显著增强了大语言模型（LLM）的推理能力，并作为一个整体引导这些模型生成通向最终答案的推理步骤。然而，我们观察到，CoT 示范中孤立的片段、词语或 token 可能意外干扰 LLM 的生成过程。模型可能过度关注示范中存在的某些局部信息，从而在推理过程中引入无关噪声，并可能导致错误答案。本文通过在每个输出步骤动态追踪和操纵 LLM 的内部运作来研究 CoT 的底层机制；结果表明，表现出特定注意力特征的 token 更有可能诱导模型断章取义；这些 token 直接注意到与预测相关的隐藏状态，而没有充分整合非局部信息。基于这些洞见，我们提出了一种少样本注意力干预方法（Few-shot Attention Intervention, FAI），该方法动态分析示范的注意力模式，以准确识别这些 token，随后对注意力权重进行有针对性的调整，从而有效抑制它们对 LLM 的干扰作用。跨多个基准的综合实验表明，该方法相较基线方法取得了持续提升，其中在 AQuA 数据集上实现了显著的 5.91% 提升，进一步凸显了 FAI 的有效性。

英文摘要：

Few-shot Chain-of-Thought (CoT) significantly enhances the reasoning capabilities of large language models (LLMs), functioning as a whole to guide these models in generating reasoning steps toward final answers. However, we observe that isolated segments, words, or tokens within CoT demonstrations can unexpectedly disrupt the generation process of LLMs. The model may overly concentrate on certain local information present in the demonstration, introducing irrelevant noise into the reasoning process and potentially leading to incorrect answers. In this paper, we investigate the underlying mechanism of CoT through dynamically tracing and manipulating the inner workings of LLMs at each output step, which demonstrates that tokens exhibiting specific attention characteristics are more likely to induce the model to take things out of context; these tokens directly attend to the hidden states tied with prediction, without substantial integration of non-local information. Building upon these insights, we propose a Few-shot Attention Intervention method (FAI) that dynamically analyzes the attention patterns of demonstrations to accurately identify these tokens and subsequently make targeted adjustments to the attention weights to effectively suppress their distracting effect on LLMs. Comprehensive experiments across multiple benchmarks demonstrate consistent improvements over baseline methods, with a remarkable 5.91% improvement on the AQuA dataset, further highlighting the effectiveness of FAI.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。