思维链提示掩盖大型语言模型中的幻觉线索：一项实证评估
Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation
AI总结：
本文实证评估了思维链（CoT）提示对大型语言模型幻觉检测的影响，发现CoT虽能降低幻觉频率，却会掩盖检测信号从而损害检测方法的有效性，揭示了推理使用中被忽视的权衡。
摘要：
大型语言模型（LLMs）经常表现出幻觉，在响应提示时生成事实错误或语义无关的内容。思维链（CoT）提示可以通过鼓励逐步推理来缓解幻觉，但其对幻觉检测的影响仍未得到充分探索。为了填补这一空白，我们进行了一项系统的实证评估。我们首先进行了一项试点实验，揭示了CoT推理会显著影响LLM的内部状态和token概率分布。在此基础上，我们评估了各种CoT提示方法对主流幻觉检测方法的影响，涵盖了指令微调和推理导向的LLM。具体而言，我们考察了三个关键维度：幻觉分数分布的变化、检测准确率的差异以及检测置信度的偏移。我们的研究结果表明，虽然CoT提示有助于降低幻觉频率，但它也往往掩盖了用于检测的关键信号，从而损害了各种检测方法的有效性。我们的研究突出了推理使用中一个被忽视的权衡。代码公开获取于：https://github.com/ECNU-Text-Computing/cot-hallu-detect 。
英文摘要：
Large Language Models (LLMs) often exhibit \textit{hallucinations}, generating factually incorrect or semantically irrelevant content in response to prompts. Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning, but its impact on hallucination detection remains underexplored. To bridge this gap, we conduct a systematic empirical evaluation. We begin with a pilot experiment, revealing that CoT reasoning significantly affects the LLM's internal states and token probability distributions. Building on this, we evaluate the impact of various CoT prompting methods on mainstream hallucination detection methods across both instruction-tuned and reasoning-oriented LLMs. Specifically, we examine three key dimensions: changes in hallucination score distributions, variations in detection accuracy, and shifts in detection confidence. Our findings show that while CoT prompting helps reduce hallucination frequency, it also tends to obscure critical signals used for detection, impairing the effectiveness of various detection methods. Our study highlights an overlooked trade-off in the use of reasoning. Code is publicly available at: https://github.com/ECNU-Text-Computing/cot-hallu-detect .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
