arXiv 2509.115362025-12-05cs.CLcs.AI

通过推理子空间投影进行幻觉检测：HARP

HARP: Hallucination Detection via Reasoning Subspace Projection

Junjie Hu, Gang Tu, ShengYu Cheng, Jinxin Li, Jinting Wang, Rui Chen, Zhilong Zhou, Dongbo Shan

发表信息：: The Fourteenth International Conference on Learning Representations (ICLR 2026)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

HARP通过推理子空间投影方法检测大语言模型中的幻觉，通过分解隐藏状态空间并应用SVD提升鲁棒性，实验显示在TriviaQA上达到92.8%的AUROC。

摘要：

大语言模型（LLMs）中的幻觉是其在关键决策中可靠应用的主要障碍。尽管现有幻觉检测方法提高了准确性，但仍难以分离语义和推理信息并保持鲁棒性。为此，我们提出HARP（通过推理子空间投影进行幻觉检测）框架。HARP证明LLMs的隐藏状态空间可分解为语义子空间和推理子空间的直和，其中前者编码语言表达，后者捕捉内部推理过程。此外，我们展示解嵌层可以分离这些子空间，并通过奇异值分解（SVD）其参数获得跨越语义和推理子空间的基向量。最后，HARP将隐藏状态投影到推理子空间的基向量上，然后将这些投影用作LLMs幻觉检测的输入特征。通过这些投影，HARP将特征维度降低到约原来的5%，过滤掉大部分噪声，并实现增强的鲁棒性。在多个数据集上的实验表明，HARP实现了最先进的幻觉检测性能；特别是在TriviaQA上，其AUROC达到92.8%，比先前最佳方法高出7.5%。

英文摘要：

Hallucinations in Large Language Models (LLMs) pose a major barrier to their reliable use in critical decision-making. Although existing hallucination detection methods have improved accuracy, they still struggle with disentangling semantic and reasoning information and maintaining robustness. To address these challenges, we propose HARP (Hallucination detection via reasoning subspace projection), a novel hallucination detection framework. HARP establishes that the hidden state space of LLMs can be decomposed into a direct sum of a semantic subspace and a reasoning subspace, where the former encodes linguistic expression and the latter captures internal reasoning processes. Moreover, we demonstrate that the Unembedding layer can disentangle these subspaces, and by applying Singular Value Decomposition (SVD) to its parameters, the basis vectors spanning the semantic and reasoning subspaces are obtained. Finally, HARP projects hidden states onto the basis vectors of the reasoning subspace, and the resulting projections are then used as input features for hallucination detection in LLMs. By using these projections, HARP reduces the dimension of the feature to approximately 5% of the original, filters out most noise, and achieves enhanced robustness. Experiments across multiple datasets show that HARP achieves state-of-the-art hallucination detection performance; in particular, it achieves an AUROC of 92.8% on TriviaQA, outperforming the previous best method by 7.5%.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。