The rise of large language models (LLMs) has significantly influenced the quality of information in decision-making systems, leading to the prevalence of AI-generated content and challenges in detecting misinformation and managing conflicting information, or "inter-evidence conflicts." This study introduces a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios. We evaluate conflict detection methods, including Natural Language Inference (NLI) models, factual consistency (FC) models, and LLMs, on these conflicts (RQ1) and analyze LLMs' conflict resolution behaviors (RQ2). Our key findings include: (1) NLI and LLM models exhibit high precision in detecting answer conflicts, though weaker models suffer from low recall; (2) FC models struggle with lexically similar answer conflicts, while NLI and LLM models handle these better; and (3) stronger models like GPT-4 show robust performance, especially with nuanced conflicts. For conflict resolution, LLMs often favor one piece of conflicting evidence without justification and rely on internal knowledge if they have prior beliefs.
Enhancing Conflict Resolution in Language Models via Abstract Argumentation
School of Philosophy, Zhejiang University(浙江大学哲学系);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);The State Key Lab of Brain-Machine Intelligence(脑机智能国家重点实验室)
Neuro-Symbolic Resolution of Recommendation Conflicts in Multimorbidity Clinical Guidelines
Peking University(北京大学);National Institute of Health Data Science(国家健康数据科学研究院);Peking University Health Science Center(北京大学医学部);Institute of Medical Technology(医学技术研究院)
Conflicts, Villains, Resolutions: Towards models of Narrative Media Framing
The University of Melbourne(墨尔本大学);The Hong Kong Polytechnic University(香港理工大学);CSIRO Data61(澳大利亚联邦科学与工业研究组织Data61);The Australian National University(澳大利亚国立大学)