探究大型语言模型中的上下文忠实性：记忆强度与证据风格的作用
Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
AI总结：
本研究探究了大型语言模型的上下文忠实性，发现记忆强度和证据呈现风格显著影响模型对外部证据的接受度，为改进检索增强生成提供了关键见解。
摘要：
检索增强生成（RAG）通过将外部信息纳入响应生成过程来改进大型语言模型（LLMs）。然而，LLMs 的上下文忠实度有多高以及哪些因素影响 LLMs 的上下文忠实度在很大程度上仍未被探索。在本研究中，我们调查了记忆强度和证据呈现方式对 LLMs 接受外部证据的影响。我们通过测量 LLMs 对同一问题不同改写版本的响应差异来量化 LLMs 的记忆强度，这是以前的工作未考虑的。我们还生成了各种风格的证据来检验 LLMs 的行为。结果表明，对于具有高记忆强度的问题，LLMs 更倾向于依赖内部记忆。此外，与简单重复或添加细节相比，呈现改写的证据显著提高了 LLMs 的接受度。这些发现为改进检索增强生成和上下文感知 LLMs 提供了关键见解。我们的代码可在 https://github.com/liyp0095/ContextFaithful 获取。
英文摘要：
Retrieval-augmented generation (RAG) improves Large Language Models (LLMs) by incorporating external information into the response generation process. However, how context-faithful LLMs are and what factors influence LLMs' context faithfulness remain largely unexplored. In this study, we investigate the impact of memory strength and evidence presentation on LLMs' receptiveness to external evidence. We quantify the memory strength of LLMs by measuring the divergence in LLMs' responses to different paraphrases of the same question, which is not considered by previous works. We also generate evidence in various styles to examine LLMs' behavior. Our results show that for questions with high memory strength, LLMs are more likely to rely on internal memory. Furthermore, presenting paraphrased evidence significantly increases LLMs' receptiveness compared to simple repetition or adding details. These findings provide key insights for improving retrieval-augmented generation and context-aware LLMs. Our code is available at https://github.com/liyp0095/ContextFaithful.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
