arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.00935cs.CL
收藏

解决大型语言模型中的知识冲突

Resolving Knowledge Conflicts in Large Language Models

University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Carnegie Mellon University(卡内基梅隆大学)

Yike Wang, Shangbin Feng, Heng Wang, Weijia Shi, Vidhisha Balachandran, Tianxing He, Yulia Tsvetkov

AI总结:

本文提出一个评估框架以模拟上下文知识冲突,发现LLMs虽能识别冲突但难以精确定位并给出不同答案,并据此提出基于指令的增强方法以提升LLMs应对知识冲突的能力。

摘要:

大型语言模型(LLMs)经常遇到知识冲突,即LLMs的内部参数化知识与提示上下文中提供的非参数化信息之间出现差异的场景。在这项工作中,我们探讨当知识冲突发生时对LLMs的期望是什么,以及现有的LLMs是否满足这些期望。我们假设LLMs应该1)识别知识冲突,2)精确定位冲突信息片段,以及3)在冲突场景中提供不同的答案或观点。为此,我们引入了一个评估框架,用于模拟上下文知识冲突,并定量评估LLMs在多大程度上实现了这些目标。该框架包括多样且复杂的知识冲突情况、来自不同实体和领域的知识、两种合成冲突创建方法,以及难度逐步增加的设置,以反映真实的知识冲突。基于该框架的大量实验表明,虽然LLMs在识别知识冲突的存在方面表现良好,但它们在确定具体的冲突知识并在冲突信息中产生包含不同答案的响应方面存在困难。为了应对这些挑战,我们提出了新的基于指令的方法,以增强LLMs更好地实现这三个目标。进一步的分析表明,处理知识冲突的能力受到知识领域等因素的极大影响,而生成对知识冲突场景的鲁棒响应仍然是一个开放的研究问题。

英文摘要:

Large language models (LLMs) often encounter knowledge conflicts, scenarios where discrepancy arises between the internal parametric knowledge of LLMs and non-parametric information provided in the prompt context. In this work we ask what are the desiderata for LLMs when a knowledge conflict arises and whether existing LLMs fulfill them. We posit that LLMs should 1) identify knowledge conflicts, 2) pinpoint conflicting information segments, and 3) provide distinct answers or viewpoints in conflicting scenarios. To this end, we introduce an evaluation framework for simulating contextual knowledge conflicts and quantitatively evaluating to what extent LLMs achieve these goals. It includes diverse and complex situations of knowledge conflict, knowledge from diverse entities and domains, two synthetic conflict creation methods, and settings with progressively increasing difficulty to reflect realistic knowledge conflicts. Extensive experiments with the framework reveal that while LLMs perform well in identifying the existence of knowledge conflicts, they struggle to determine the specific conflicting knowledge and produce a response with distinct answers amidst conflicting information. To address these challenges, we propose new instruction-based approaches that augment LLMs to better achieve the three goals. Further analysis shows that abilities to tackle knowledge conflicts are greatly impacted by factors such as knowledge domain, while generating robust responses to knowledge conflict scenarios remains an open research question.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑