arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.08346cs.CLcs.AI
收藏

长形式幻觉检测的合理性检查

Sanity Checks for Long-Form Hallucination Detection

Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of British Columbia(不列颠哥伦比亚大学)

Geigh Zollicoffer, Minh Vu, Hongli Zhan, Raymond Li, Manish Bhattarai

AI总结:

本文提出一种控制不变性方法,通过强制和移除测试区分推理本身与最终答案的表面关联,展示轻量级评分器TRACT在未扰动轨迹上的有效性。

摘要:

大型语言模型的幻觉检测方法越来越多地基于链式推理轨迹,但尚不清楚它们是评估推理本身还是仅利用最终答案的表面相关性。我们引入了一种受控不变性方法,通过两个 oracle 测试:Force,将每个响应的最终答案替换为地面真实值同时保留推理轨迹,和 Remove,移除答案宣布步骤同时保留轨迹。这揭示了它们的预测能力是否源于答案层面的伪影,而非中间推理的结构或有效性。我们进一步表明,一旦这些伪影被控制,有效的检测不一定需要复杂的学得表示:TRACT,一个基于词汇轨迹特征(让步趋势、步骤长度动态和跨响应词汇收敛)的轻量级评分器,在未扰动轨迹上表现出强大的鲁棒性,同时在与现有基线竞争或超越上表现优异。这些发现表明,当前在推理意识幻觉检测中的主要挑战不是轨迹中信号的缺失,而是未能从终点提示中隔离信号。

英文摘要:

Hallucination detection methods for large language models increasingly operate on chain-of-thought reasoning traces, yet it remains unclear whether they evaluate the reasoning itself or merely exploit surface correlates of the final answer. We introduce a controlled-invariance methodology that exposes this distinction through two oracle tests: \textsc{Force}, which replaces each response's final answer with the ground truth while preserving the reasoning trace, and \textsc{Remove}, which strips answer-announcement steps while leaving the trajectory intact. This reveals if their predictive power derives from answer-level artifacts rather than from the structure or validity of intermediate reasoning. We further show that once these artifacts are controlled for, effective detection does not necessarily require complex learned representations: TRACT, a lightweight scorer built on lexical trajectory features (hedging trends, step-length dynamics, and cross-response vocabulary convergence), achieves strong robustness while remaining competitive with or outperforming existing baselines on unperturbed traces. These findings suggest that the current central challenge in reasoning-aware hallucination detection is not the absence of signal in the trace, but the failure to isolate it from endpoint cues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑