使用真实文档研究大语言模型在上下文-记忆冲突下的行为
Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents
AI总结:
该研究提出真实场景下的大语言模型上下文-记忆知识冲突研究框架,用真实冲突文档更新错误参数知识,发现更新失败率低于此前合成文档场景的结果,且错误参数答案会提升更新失败概率,揭示参数知识对阅读能力的负面影响。
摘要:
检索增强生成(RAG,Retrieval-augmented generation)缓解了全参数语言模型的诸多问题,例如时效性衰减、幻觉以及缺乏事实依据。在RAG中,模型的知识可通过上下文中提供的文档进行更新,这会引发模型参数知识与上下文信息之间的冲突情况,此时模型未必总能更新自身知识。此前的研究通过构造与模型正确参数答案相矛盾的合成文档,来研究上下文-记忆知识冲突。我们提出了一个在真实场景下研究此类知识冲突的框架,利用真实的冲突文档来更新错误的参数知识,这反映了实践中知识冲突的实际产生方式。在这一真实场景中,我们发现知识更新失败的频率低于此前报告的结果。在模型仍无法更新答案的情况中,我们发现了一种参数偏差:上下文中出现的错误参数答案会使知识更新更有可能失败。这些结果表明,大语言模型(LLMs)的事实性参数知识会对其阅读能力和行为产生负面影响。我们的代码可在https://github.com/kortukov/realistic_knowledge_conflicts/ 获取。
英文摘要:
Retrieval-augmented generation (RAG) mitigates many problems of fully parametric language models, such as temporal degradation, hallucinations, and lack of grounding. In RAG, the model's knowledge can be updated from documents provided in context. This leads to cases of conflict between the model's parametric knowledge and the contextual information, where the model may not always update its knowledge. Previous work studied context-memory knowledge conflicts by creating synthetic documents that contradict the model's correct parametric answers. We present a framework for studying such knowledge conflicts in a realistic setup. We update incorrect parametric knowledge using real conflicting documents. This reflects how knowledge conflicts arise in practice. In this realistic scenario, we find that knowledge updates fail less often than previously reported. In cases where the models still fail to update their answers, we find a parametric bias: the incorrect parametric answer appearing in context makes the knowledge update likelier to fail. These results suggest that the factual parametric knowledge of LLMs can negatively influence their reading abilities and behaviors. Our code is available at https://github.com/kortukov/realistic_knowledge_conflicts/ .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。