适应性激活取消：用于大语言模型幻觉抑制的适应性激活取消
Adaptive Activation Cancellation for Hallucination Mitigation in Large Language Models
AI总结：
提出适应性激活取消方法，通过实时干预减少大语言模型的幻觉问题，同时保持模型性能和生成质量。
摘要：
大语言模型经常生成流畅但事实错误的文本。我们提出了适应性激活取消（AAC），一种实时推理框架，将与幻觉相关的神经激活视为Transformer残差流中的结构干扰，借鉴经典自适应噪声消除的类比。该框架通过逐层线性探测识别幻觉节点（H-Nodes），并在自回归生成过程中使用置信度加权的前向钩子来抑制它们--无需外部知识，无需微调，也不需要额外的推理过程。在TruthfulQA和HaluEval上评估了OPT-125M、Phi-3-mini和LLaMA 3-8B，实时钩子是唯一在三个规模上都能持续提高下游准确性的干预措施。关键的是，该方法是严格手术的：在所有三个模型规模上，WikiText-103困惑度和MMLU推理准确率均保持在0.0%的降级，这一特性使AAC区别于那些以牺牲流畅度或通用能力为代价来提高事实准确性的方法。在LLaMA 3-8B规模上，钩子还带来了积极的生成层面收益（MC1 +0.04；MC2 +0.003；Token-F1 +0.003），同时在探测空间选择性方面比ITI基线高5.94倍至3.5倍--证明了针对神经元层面的抑制可以同时提高事实准确性并保持模型能力。
英文摘要：
Large Language Models frequently generate fluent but factually incorrect text. We propose Adaptive Activation Cancellation (AAC), a real-time inference-time framework that treats hallucination-associated neural activations as structured interference within the transformer residual stream, drawing an explicit analogy to classical adaptive noise cancellation from signal processing. The framework identifies Hallucination Nodes (H-Nodes) via layer-wise linear probing and suppresses them using a confidence-weighted forward hook during auto-regressive generation -- requiring no external knowledge, no fine-tuning, and no additional inference passes. Evaluated across OPT-125M, Phi-3-mini, and LLaMA 3-8B on TruthfulQA and HaluEval, the real-time hook is the only intervention that consistently improves downstream accuracy on all three scales. Critically, the method is strictly surgical: WikiText-103 perplexity and MMLU reasoning accuracy are preserved at exactly 0.0% degradation across all three model scales, a property that distinguishes AAC from interventions that trade fluency or general capability for factual improvement. On the LLaMA 3-8B scale, the hook additionally yields positive generation-level gains (MC1 +0.04; MC2 +0.003; Token-F1 +0.003) while achieving probe-space selectivity 5.94x - 3.5x higher than the ITI baseline -- demonstrating that targeted neuron-level suppression can simultaneously improve factual accuracy and preserve model capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
