信任还是不信任？增强大型语言模型对外部上下文的情境忠实度
To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
AI总结：
本文研究 LLM 在 RAG 等场景中盲目信任错误外部上下文的问题，提出 SCR、RCR 及 CR-DPO 微调方法，以根据内外部知识置信度动态解决冲突并提升问答准确性。
摘要：
大型语言模型（LLM）通常会通过外部上下文进行增强，例如检索增强生成（RAG）中使用的上下文。然而，这些上下文可能不准确或被故意误导，从而与模型的内部知识发生冲突。我们认为，稳健的 LLM 应表现出情境忠实度，即根据其对内部知识和外部上下文的置信度动态校准对外部信息的信任，以解决知识冲突。为了对这种能力进行基准测试，我们在多个问答数据集上评估了 LLM，其中包括一个新创建的数据集，该数据集以来自 Reddit 帖子的真实野外错误上下文为特色。我们表明，当同时提供正确和错误上下文时，开源模型和专有模型都倾向于过度依赖外部信息，而不管其事实准确性如何。为了增强情境忠实度，我们提出了两种方法：自引导置信度推理（SCR）和基于规则的置信度推理（RCR）。SCR 使模型能够相对于自身内部知识自我评估外部信息的置信度，从而产生最准确的答案。相比之下，RCR 从 LLM 中提取显式置信度信号，并使用预定义规则确定最终答案。我们的结果表明，对于具有强推理能力的 LLM，例如 GPT-4o 和 GPT-4o mini，SCR 优于 RCR，相比直接输入增强基线最高实现 24.2% 的提升。相反，对于像 Llama-3-8B 这样的较小模型，RCR 优于 SCR。使用我们提出的置信度推理直接偏好优化（CR-DPO）方法对 SCR 进行微调，可以提高在已见和未见数据集上的性能，在 Llama-3-8B 上平均提升 8.9%。除了定量结果外，我们还提供了关于 SCR 和 RCR 相对优势的见解。
英文摘要：
Large Language Models (LLMs) are often augmented with external contexts, such as those used in retrieval-augmented generation (RAG). However, these contexts can be inaccurate or intentionally misleading, leading to conflicts with the model's internal knowledge. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context to resolve knowledge conflicts. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-assess the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B. In addition to quantitative results, we offer insights into the relative strengths of SCR and RCR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
