PARALLAX：区分真实幻觉检测与基准构建人工制品
PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts
AI总结：
本文研究了大型语言模型幻觉检测中的基准构建人工制品问题，提出DRIFT作为对比方法，发现大部分基线方法在控制条件下表现接近随机，而SAPLMA和DRIFT作为上层隐藏状态的监督探针表现出例外。
摘要：
大型语言模型（LLMs）在生成输出时常常表现出自信的幻觉：其输出可以流畅、权威且错误。在医疗、法律和科学应用中，这种失败会造成直接伤害，而通过内部模型状态检测幻觉则为更安全的部署提供了路径。越来越多的研究表明，这一问题变得越来越可处理，最近的方法在广泛使用的基准上实现了高检测性能。然而，我们发现，这种明显的进步在仔细审视后并不成立。六个语料库中的四个直接将真实答案嵌入输入提示中。我们提出的名为 extsc{TxTemb}的简单文本相似度基线利用这一点，无需访问模型内部状态即可实现接近完美的检测分数。为了衡量在消除这些人工制品后剩余的真实检测能力，我们进行了涵盖22种检测方法、12种开源模型（涵盖6种架构家族）和6个语料库的大规模评估。我们进一步引入 extbf{DRIFT}，作为实时生成检测的比较点。我们的发现表明，该领域报告的幻觉检测进展在很大程度上是由广泛使用的语料库中的基准构建人工制品所解释的；在受控条件下，大多数已建立的基线方法表现接近随机；一致的例外是SAPLMA和DRIFT，两者都是基于上层隐藏状态的监督探针。
英文摘要：
Large language models (LLMs) hallucinate with confidence: their outputs can be fluent, authoritative, and simply wrong. In medical, legal, and scientific applications this failure causes direct harm, and detecting it from internal model states offers a path to safer deployment. A growing body of work reports that this problem is increasingly tractable, with recent methods achieving high detection performance on widely used benchmarks. We show, however, that much of this apparent progress does not survive scrutiny. Four of the six corpora embed the ground-truth answer directly in the input prompt. A naïve text-similarity baseline we call \textsc{TxTemb} exploits this to achieve near-perfect detection scores without any access to model internals. To measure what genuine detection capability remains once these artifacts are controlled, we conduct a large-scale evaluation spanning twenty-two detection methods, twelve open-source models spanning six architectural families, and six corpora. We further introduce \textbf{DRIFT}, a supervised probe over inter-layer hidden-state transitions, as a point of comparison for live-generation detection. Our findings suggest that the field's reported progress on hallucination detection is substantially explained by benchmark construction artifacts in widely used corpora, and that the majority of established baselines perform near chance under controlled conditions; the consistent exceptions are SAPLMA and DRIFT, both supervised probes on upper-layer hidden states.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
