Delta——对比解码缓解大型语言模型中的文本幻觉
Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Models
AI总结：
本文提出推理时方法 Delta，通过随机遮蔽提示并对比原始与遮蔽输入的输出分布来抑制 LLM 幻觉，在多个问答基准上显著提升可靠性。
摘要：
大型语言模型（LLM）在自然语言处理中表现出强大能力，但仍容易产生幻觉，即生成事实错误或捏造的内容。这一问题削弱了其可靠性，尤其是在医疗保健和法律咨询等高风险领域。为应对该挑战，我们提出 Delta，这是一种推理时方法，无需重新训练模型或额外数据即可减少幻觉。Delta 通过随机遮蔽输入提示的部分内容，并对比原始输入与遮蔽输入的输出分布来工作，从而仅借助推理计算有效抑制幻觉。我们在上下文丰富的问答基准上评估 Delta，在 SQuAD v1.1 和 v2 上分别取得约 3 个和 6 个百分点的绝对提升，并在欠采样解码下的 TriviaQA 和 Natural Questions 上分别取得 7 个和 2 个百分点的提升。Delta 还将 SQuAD v2 上的无答案精确匹配分数提高超过 10 个百分点，证明了其在缓解由上下文歧义引起的幻觉方面的有效性。这些结果凸显 Delta 是一种计算高效且可扩展的方法，可提升 LLM 在现实世界应用中的可靠性。
英文摘要：
Large language models (LLMs) demonstrate strong capabilities in natural language processing but remain prone to hallucinations, generating factually incorrect or fabricated content. This issue undermines their reliability, particularly in high-stakes domains such as healthcare and legal advisory. To address this challenge, we propose Delta, an inference-time method that reduces hallucinations without requiring model retraining or additional data. Delta works by randomly masking parts of the input prompt and contrasting the output distributions for the original and masked inputs, effectively suppressing hallucinations through inference-only computations. We evaluate Delta on context-rich question-answering benchmarks, achieving absolute improvements of approximately 3 and 6 percentage points on SQuAD v1.1 and v2, respectively, and 7 and 2 percentage points on TriviaQA and Natural Questions under-sampling decoding. Delta also improves the no-answer exact match score on SQuAD v2 by over ten percentage points, demonstrating its effectiveness in mitigating hallucinations arising from contextual ambiguity. These results highlight Delta as a computationally efficient and scalable approach for improving the reliability of LLMs in real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
