推理去噪器：用于大型推理模型中幻觉检测的推理痕迹去噪

中文摘要 AI 辅助

大型推理模型（LRMs）在给出最终答案前会生成很长的推理痕迹。这些痕迹可能包含用于幻觉检测的有用信号，但利用它们并非易事，因为长轨迹常包含噪声步骤，掩盖了与真实性评估相关的线索。本文识别出两种常见的推理噪声形式，即无关步骤和重复步骤，并表明它们都会大幅降低幻觉检测性能。现有的基于置信度的分数和基于朴素嵌入的过滤无法可靠地区分噪声和信息步骤。为应对这一挑战，我们提出了REDE，一种用于幻觉检测的推理痕迹去噪的新型学习框架。具体而言，REDE利用最终答案注意力作为自动监督信号来塑造步骤级表示空间，产生可可靠识别和过滤噪声步骤的精细嵌入。通过对去除噪声步骤后的过滤推理轨迹进行操作，REDE可以很容易地插入到各种幻觉检测器中。在多个推理基准上的大量实验表明，REDE始终比有竞争力的基线提高检测性能。

英文摘要

Large reasoning models (LRMs) generate long reasoning traces before producing final answers. While these traces may contain useful signals for hallucination detection, harnessing them is non-trivial because long trajectories often include noisy steps that obscure the cues relevant to truthfulness assessment. In this paper, we identify two prevalent forms of reasoning noises, i.e., irrelevant steps and repetitive steps, and show that both substantially degrade hallucination detection performance. Existing confidence-based scores and naive embedding-based filtering fail to reliably separate noisy from informative steps. To address this challenge, we propose REDE, a novel learning framework for denoising reasoning traces for hallucination detection. Specifically, REDE leverages final-answer attention as an automatic supervision signal to shape the step-level representation space, yielding refined embeddings in which noisy steps can be reliably identified and filtered. REDE can be readily plugged into diverse hallucination detectors by operating on the filtered reasoning trajectory after removing noisy steps. Extensive experiments on multiple reasoning benchmarks show that REDE consistently improves detection performance over competitive baselines.

发表机构

• College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
• Zhejiang University（浙江大学）
• Australian Artificial Intelligence Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）

机构由 AI 辅助整理，请以论文原文为准。