HICD：通过注意力分散诱导幻觉用于对比解码以缓解大语言模型中的幻觉
HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models
AI总结：
提出HICD方法，通过选择关键注意力头并分散其注意力诱导幻觉，与原始输出对比解码，以缓解大语言模型幻觉，提升上下文忠实性和事实性任务性能。
摘要：
大语言模型（LLMs）经常产生幻觉，生成上下文不准确或事实错误的输出。我们提出了HICD，一种新颖的方法，旨在为对比解码诱导幻觉以缓解幻觉。与现有的对比解码方法不同，HICD选择对模型预测至关重要的注意力头作为诱导头，然后通过分散这些诱导头的注意力来诱导幻觉，并将幻觉输出与原始输出进行比较以获得最终结果。我们的方法在需要上下文忠实性的任务上显著提升了性能，例如上下文补全、阅读理解和问答。它还在需要准确知识回忆的任务中提高了事实性。我们证明，我们的诱导头选择和注意力分散方法为对比解码产生了更具“对比有效性”的幻觉，优于其他幻觉诱导方法。我们的发现提供了一种有前景的策略，通过以受控方式诱导幻觉来减少幻觉，从而在广泛的任务中增强LLMs的性能。
英文摘要：
Large Language Models (LLMs) often generate hallucinations, producing outputs that are contextually inaccurate or factually incorrect. We introduce HICD, a novel method designed to induce hallucinations for contrastive decoding to mitigate hallucinations. Unlike existing contrastive decoding methods, HICD selects attention heads crucial to the model's prediction as inducing heads, then induces hallucinations by dispersing attention of these inducing heads and compares the hallucinated outputs with the original outputs to obtain the final result. Our approach significantly improves performance on tasks requiring contextual faithfulness, such as context completion, reading comprehension, and question answering. It also improves factuality in tasks requiring accurate knowledge recall. We demonstrate that our inducing heads selection and attention dispersion method leads to more "contrast-effective" hallucinations for contrastive decoding, outperforming other hallucination-inducing methods. Our findings provide a promising strategy for reducing hallucinations by inducing hallucinations in a controlled manner, enhancing the performance of LLMs in a wide range of tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
