跨层注意力探测用于细粒度幻觉检测
Cross-Layer Attention Probing for Fine-Grained Hallucination Detection
AI总结：
本文提出跨层注意力探测（CLAP）技术，通过处理LLM残差流激活实现细粒度幻觉检测，并基于此构建“先检测后缓解”策略以提升模型可靠性。
摘要：
随着大型语言模型（LLMs）在各种应用中的大规模采用，由于其倾向于生成不准确文本（即幻觉），可靠性问题日益凸显。在这项工作中，我们提出了跨层注意力探测（CLAP），一种用于幻觉检测的新型激活探测技术，它将整个残差流中的 LLM 激活作为联合序列进行处理。我们使用五个 LLM 和三个任务进行的实证评估表明，与基线相比，CLAP 在贪婪解码响应以及在更高温度下采样的响应上均改进了幻觉检测，从而实现了细粒度检测，即对给定提示的不同采样响应中的幻觉和非幻觉进行消歧的能力。这使我们能够提出一种使用 CLAP 的“先检测后缓解”策略，与直接缓解方法相比，该策略能减少幻觉并提高 LLM 可靠性。最后，我们表明 CLAP 即使在分布外应用时也能保持高可靠性。
英文摘要：
With the large-scale adoption of Large Language Models (LLMs) in various applications, there is a growing reliability concern due to their tendency to generate inaccurate text, i.e. hallucinations. In this work, we propose Cross-Layer Attention Probing (CLAP), a novel activation probing technique for hallucination detection, which processes the LLM activations across the entire residual stream as a joint sequence. Our empirical evaluations using five LLMs and three tasks show that CLAP improves hallucination detection compared to baselines on both greedy decoded responses as well as responses sampled at higher temperatures, thus enabling fine-grained detection, i.e. the ability to disambiguate hallucinations and non-hallucinations among different sampled responses to a given prompt. This allows us to propose a detect-then-mitigate strategy using CLAP to reduce hallucinations and improve LLM reliability compared to direct mitigation approaches. Finally, we show that CLAP maintains high reliability even when applied out-of-distribution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
