抽象激活空间用于大语言模型中的内容不变推理
Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models
AI总结：
本研究提出了一种抽象引导推理框架，通过分离结构推理与词法语义，减少语义干扰对形式推理的影响，提升大语言模型的推理鲁棒性。
摘要：
大型语言模型（LLMs）在演绎判断中常常遇到困难，系统性地将语义可能性与形式有效性混淆，这种现象被称为内容效应。这种偏差即使在模型生成逐步解释时仍然存在，表明中间推理可能继承了影响答案的相同语义捷径。最近的解决方案提出通过增加推理时间的结构约束来缓解这一问题，要么通过鼓励抽象的中间表示，要么直接干预模型的内部计算；然而，可靠地抑制语义干扰仍然是一个开放的挑战。为了使形式推理对语义内容更不敏感，我们引入了一个引导抽象推理的框架，明确将结构推理与词法语义分开。我们构建了配对的内容丰富和抽象三段论，并使用模型在抽象输入上的激活来定义抽象推理空间。然后，我们学习轻量级的抽象器，从内容条件的残差流状态中预测与该空间对齐的表示，并在正向传递过程中通过多层干预整合这些预测。使用跨语言迁移作为测试平台，我们证明了与抽象对齐的引导减少了由内容驱动的错误，并提高了对有效性敏感的性能。我们的结果将激活层面的抽象定位为一种可扩展的机制，用于增强LLMs在对抗语义干扰方面的推理鲁棒性。
英文摘要：
Large Language Models (LLMs) often struggle with deductive judgment in syllogistic reasoning, systematically conflating semantic plausibility with formal validity a phenomenon known as content effect. This bias persists even when models generate step-wise explanations, indicating that intermediate rationales may inherit the same semantic shortcuts that affect answers. Recent approaches propose mitigating this issue by increasing inference-time structural constraints, either by encouraging abstract intermediate representations or by intervening directly in the model's internal computations; however, reliably suppressing semantic interference remains an open challenge. To make formal deduction less sensitive to semantic content, we introduce a framework for abstraction-guided reasoning that explicitly separates structural inference from lexical semantics. We construct paired content-laden and abstract syllogisms and use the model's activations on abstract inputs to define an abstract reasoning space. We then learn lightweight Abstractors that, from content-conditioned residual-stream states, predict representations aligned with this space and integrate these predictions via multi-layer interventions during the forward pass. Using cross-lingual transfer as a test bed, we show that abstraction-aligned steering reduces content-driven errors and improves validity-sensitive performance. Our results position activation-level abstraction as a scalable mechanism for enhancing the robustness of formal reasoning in LLMs against semantic interference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
