通过注意力感知干预改进逻辑推理的链式思维
Improving Chain-of-Thought for Logical Reasoning via Attention-Aware Intervention
AI总结：
通过注意力感知干预改进逻辑推理的链式思维，提出一种非交互式端到端框架，提升模型推理能力与可分析性，无需外部资源。
摘要：
现代基于大语言模型的逻辑推理主要依赖于采用复杂的交互框架，将推理过程分解为通过精心设计的提示或需要外部资源（例如符号求解器）解决的子任务，以利用其强大的逻辑结构。尽管交互式方法引入了额外的开销或依赖于外部组件，从而限制了其可扩展性。在本工作中，我们介绍了一个非交互式、端到端的推理框架，使推理能够在模型自身中产生，从而在不使用任何外部资源的情况下提高泛化能力并保持可分析性。我们证明了在少量示例提示中引入结构信息会激活一组与逻辑推理运算符对齐的注意力头。基于这一见解，我们提出了注意力感知干预（AAI），一种推理时的干预方法，通过重新加权选定头的注意力分数来调整注意力得分。AAI提供了一种高效的方法，通过注意力调节引导模型的推理过程利用先验知识。广泛的实验表明，AAI在各种基准和模型架构上均提高了逻辑推理性能，同时不会引入显著的额外计算开销。代码可在https://github.com/phuongnm94/aai_for_logical_reasoning上获得。
英文摘要：
Modern logical reasoning with LLMs primarily relies on employing complex interactive frameworks that decompose the reasoning process into subtasks solved through carefully designed prompts or requiring external resources (e.g., symbolic solvers) to exploit their strong logical structures. While interactive approaches introduce additional overhead or depend on external components, which limit their scalability. In this work, we introduce a non-interactive, end-to-end framework for reasoning tasks, enabling reasoning to emerge within the model itself-improving generalization while preserving analyzability without any external resources. We show that introducing structural information into the few-shot prompt activates a subset of attention heads that patterns aligned with logical reasoning operators. Building on this insight, we propose Attention-Aware Intervention (AAI), an inference-time intervention method that reweights attention scores across selected heads identified by their logical patterns. AAI offers an efficient way to steer the model's reasoning toward leveraging prior knowledge through attention modulation. Extensive experiments show that AAI enhances logical reasoning performance across diverse benchmarks, and model architectures, while incurring negligible additional computational overhead. Code is available at https://github.com/phuongnm94/aai_for_logical_reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
