TriLens: 基于逐层Logit-Lens熵的白盒幻觉检测
TriLens: Per-Layer Logit-Lens Entropy for White-Box Hallucination Detection
AI总结：
提出TriLens方法，通过在每个Transformer层读取多头自注意力、前馈网络和残差流的logit-lens输出熵，构建紧凑的3L维轨迹，有效检测大语言模型幻觉。
摘要：
当语言模型产生幻觉时，最终答案是错误的，但错误在模型内部并非不可见。不同的内部路径可能保持不确定，在锐化速度上不一致，或在输出产生前承诺相互竞争的延续。我们提出TriLens，一种白盒检测器，将这一直觉转化为紧凑表示：在每一层，它通过模型自身的logit透镜读取多头自注意力输出、前馈输出和残差流，然后仅记录每个读出的熵。得到的3L维轨迹描述了确定性如何跨深度和跨模块形成，无需存储高维隐藏状态或采样多个生成。这一简单信号在指令微调LLM和QA基准测试中产生了强大的检测器，我们的分析表明，三个模块的熵轨迹提供了互补证据。TriLens表明，幻觉检测可以从跟踪内部计算如何稳定中受益，而不仅仅是最终层的预测。
英文摘要：
When a language model hallucinates, the final answer is wrong, but the mistake is not necessarily invisible inside the model. Different internal pathways may remain uncertain, disagree in how quickly they sharpen, or commit to competing continuations before the output is produced. We introduce TriLens, a white-box detector that turns this intuition into a compact representation: at every layer, it reads the multi-head self-attention output, the feed-forward output, and the residual stream through the model's own logit lens, then records only the entropy of each readout. The resulting 3L-dimensional trajectory describes how certainty forms across depth and across modules, without storing high-dimensional hidden states or sampling multiple generations. This simple signal yields a strong detector across instruction-tuned LLMs and QA benchmarks, and our analyses show that the three module-wise entropy trajectories provide complementary evidence. TriLens suggests that hallucination detection can benefit from tracking how internal computation settles, not only what the final layer predicts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
