arXiv 2604.164302026-04-06cs.CLcs.AI

HalluSAE：通过稀疏自编码器检测大型语言模型中的幻觉

HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders

Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进研究院，北京航空航天大学人工智能学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）；Renmin University of China（中国人民大学）

Boshui Chen, Zhaoxin Fan, Ke Wang, Zhiying Leng, Faguo Wu, Hongwei Zheng, Yifan Sun, Wenjun Wu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

HalluSAE通过稀疏自编码器和几何势能度量定位潜在相变区域，利用对比logit归因识别幻觉相关稀疏特征，通过线性探针检测因果幻觉，实验表明其在Gemma-2-9B上达到最先进的幻觉检测性能。

摘要：

大型语言模型（LLMs）强大且广泛采用，但其实际影响受限于已知的幻觉现象。尽管最近的幻觉检测方法取得了显著进展，但大多数方法忽略了其动态性质和潜在机制。为解决这一差距，我们提出了HalluSAE，一种受相变启发的框架，将幻觉建模为模型潜在动态中的关键转变。通过将生成过程建模为通过势能景观的轨迹，HalluSAE识别关键转变区域，并将事实错误归因于特定高能稀疏特征。我们的方法包括三个阶段：（1）通过稀疏自编码器和几何势能度量进行势能赋能的相变区域定位；（2）利用对比logit归因进行幻觉相关的稀疏特征归因；（3）通过在线性探针上解耦特征上进行基于探针的因果幻觉检测。在Gemma-2-9B上的大量实验表明，HalluSAE在幻觉检测性能上达到最先进的水平。

英文摘要：

Large Language Models (LLMs) are powerful and widely adopted, but their practical impact is limited by the well-known hallucination phenomenon. While recent hallucination detection methods have made notable progress, we find most of them overlook the dynamic nature and underlying mechanisms of it. To address this gap, we propose HalluSAE, a phase transition-inspired framework that models hallucination as a critical shift in the model's latent dynamics. By modeling the generation process as a trajectory through a potential energy landscape, HalluSAE identifies critical transition zones and attributes factual errors to specific high-energy sparse features. Our approach consists of three stages: (1) Potential Energy Empowered Phase Zone Localization via sparse autoencoders and a geometric potential energy metric; (2) Hallucination-related Sparse Feature Attribution using contrastive logit attribution; and (3) Probing-based Causal Hallucination Detection through linear probes on disentangled features. Extensive experiments on Gemma-2-9B demonstrate that HalluSAE achieves state-of-the-art hallucination detection performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。