arXiv 2509.065962025-09-08cs.CLcs.AI

HAVE：面向大语言模型幻觉缓解的头部自适应门控与值校准

HAVE: Head-Adaptive Gating and ValuE Calibration for Hallucination Mitigation in Large Language Models

Xin Tong School of Information and Cyber Security People’s Public Security University of China（信息与网络安全学院中国人民公安大学）；Zhi Lin School of Safety Science Tsinghua University（安全科学学院清华大学）；Jingya Wang School of Information and Cyber Security People’s Public Security University of China（信息与网络安全学院中国人民公安大学）；Bo Jin* The Third Research Institute of the Ministry of Public Security of China（公安部第三研究所）

Xin Tong, Zhi Lin, Jingya Wang, Bo Jin

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大语言模型在检索增强或长上下文生成中的幻觉问题，提出无需参数和微调的HAVE解码框架，通过头部自适应门控与值校准融合证据，在多个问答基准上优于强基线。

摘要：

大型语言模型（LLMs）在检索增强或长上下文生成中经常产生幻觉，即使存在相关证据时也是如此。这源于两个问题：头部重要性被视为与输入无关，且原始注意力权重难以准确反映每个标记的真实贡献。我们提出HAVE（头部自适应门控与值校准），一种无需参数的解码框架，直接解决上述两个挑战。HAVE引入头部自适应门控，对注意力头部进行实例级软重新加权，以及值校准，利用值向量的幅度增强注意力以近似写回贡献。这些模块共同构建与模型更新对齐的标记级证据，并通过轻量级不确定性缩放策略将其与语言模型分布融合。HAVE无需微调，仅需单次前向传播即可运行，高效且广泛适用。在多个问答基准和LLM系列上的实验表明，HAVE持续减少幻觉，并以适度开销优于包括DAGCD在内的强基线。该框架透明、可复现，易于与现成LLM集成，推动真实场景中的可信生成。

英文摘要：

Large Language Models (LLMs) often produce hallucinations in retrieval-augmented or long-context generation, even when relevant evidence is present. This stems from two issues: head importance is treated as input-agnostic, and raw attention weights poorly reflect each token's true contribution. We present HAVE (Head-Adaptive Gating and ValuE Calibration), a parameter-free decoding framework that directly addresses both challenges. HAVE introduces head-adaptive gating, which performs instance-level soft reweighing of attention heads, and value calibration, which augments attention with the magnitude of value vectors to approximate write-back contribution. Together, these modules construct token-level evidence aligned with model updates and fuse it with the LM distribution through a lightweight uncertainty-scaled policy. HAVE requires no finetuning and operates in a single forward pass, making it efficient and broadly applicable. Experiments across multiple QA benchmarks and LLM families demonstrate that HAVE consistently reduces hallucinations and outperforms strong baselines, including DAGCD, with modest overhead. The framework is transparent, reproducible, and readily integrates with off-the-shelf LLMs, advancing trustworthy generation in real-world settings.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。