Large language models (LLMs) often encounter knowledge conflicts, scenarios where discrepancy arises between the internal parametric knowledge of LLMs and non-parametric information provided in the prompt context. In this work we ask what are the desiderata for LLMs when a knowledge conflict arises and whether existing LLMs fulfill them. We posit that LLMs should 1) identify knowledge conflicts, 2) pinpoint conflicting information segments, and 3) provide distinct answers or viewpoints in conflicting scenarios. To this end, we introduce an evaluation framework for simulating contextual knowledge conflicts and quantitatively evaluating to what extent LLMs achieve these goals. It includes diverse and complex situations of knowledge conflict, knowledge from diverse entities and domains, two synthetic conflict creation methods, and settings with progressively increasing difficulty to reflect realistic knowledge conflicts. Extensive experiments with the framework reveal that while LLMs perform well in identifying the existence of knowledge conflicts, they struggle to determine the specific conflicting knowledge and produce a response with distinct answers amidst conflicting information. To address these challenges, we propose new instruction-based approaches that augment LLMs to better achieve the three goals. Further analysis shows that abilities to tackle knowledge conflicts are greatly impacted by factors such as knowledge domain, while generating robust responses to knowledge conflict scenarios remains an open research question.
Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuning
Department of Gastroenterology, Renmin Hospital, Wuhan University(消化内科,仁医医院,武汉大学);School of Computer Science, Wuhan University(计算机学院,武汉大学);School of Computer Science, Faculty of Engineering, The University of Sydney(计算机学院,工程学院,悉尼大学)
ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
Institute of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学)
Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
School of Computer Science and Technology, Xi'an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Computer Science, The University of Manchester(曼彻斯特大学计算机科学系);Hunan University(湖南大学);National University of Singapore(新加坡国立大学)
Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning
Hong Kong University of Science and Technology(香港科技大学);University of Southern California(南加州大学);Tencent AI Lab(腾讯AI实验室);University of California, Davis(加州大学戴维斯分校)
Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Tencent(腾讯);Renmin University of China(中国人民大学);Johns Hopkins University(约翰霍普金斯大学)
Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMs
King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);University of Macau(澳门大学);Zhejiang University(浙江大学);Tongji University(同济大学);University of Hamburg(汉堡大学);South China University of Technology(华南理工大学)
本文提出辩证对齐框架,利用 AI 反馈构建训练数据,以提升大语言模型对投毒上下文攻击的防御能力,同时保持上下文知识编辑效果,无需额外提示工程。
FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation
School of Informatics, Xiamen University(厦门大学信息学院);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);The Hong Kong Polytechnic University(香港理工大学);Migu Meland Co., Ltd(Migu Meland公司);Soochow University(苏州大学);Singapore Management University(新加坡管理大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
KaFT: Knowledge-aware Fine-tuning for Boosting LLMs' Domain-specific Question-Answering Performance
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence and Hubei Key Laboratory of Multimedia and Network Communication Engineering(计算机科学学院、多媒体软件国家工程研究中心、人工智能研究院、多媒体与网络通信工程湖北省重点实验室)
ArbESC+: Arabic Enhanced Edit Selection System Combination for Grammatical Error Correction Resolving conflict and improving system combination in Arabic GEC
Department of Computer Sciences, Faculty of Computing and Information Technology(计算机科学系,计算与信息技术学院);King Abdulaziz University(国王阿卜杜勒阿齐兹大学);Saudi Electronic University(沙特电子大学)