The spread of misinformation across digital platforms can pose significant societal risks. Claim verification, a.k.a. fact-checking, systems can help identify potential misinformation. However, their efficacy is limited by the knowledge sources that they rely on. Most automated claim verification systems depend on a single knowledge source and utilize the supporting evidence from that source; they ignore the disagreement of their source with others. This limits their knowledge coverage and transparency. To address these limitations, we present a novel system for open-domain claim verification (ODCV) that leverages large language models (LLMs), multi-perspective evidence retrieval, and cross-source disagreement analysis. Our approach introduces a novel retrieval strategy that collects evidence for both the original and the negated forms of a claim, enabling the system to capture supporting and contradicting information from diverse sources: Wikipedia, PubMed, and Google. These evidence sets are filtered, deduplicated, and aggregated across sources to form a unified and enriched knowledge base that better reflects the complexity of real-world information. This aggregated evidence is then used for claim verification using LLMs. We further enhance interpretability by analyzing model confidence scores to quantify and visualize inter-source disagreement. Through extensive evaluation on four benchmark datasets with five LLMs, we show that knowledge aggregation not only improves claim verification but also reveals differences in source-specific reasoning. Our findings underscore the importance of embracing diversity, contradiction, and aggregation in evidence for building reliable and transparent claim verification systems
UNC Charlotte（北卡罗来纳大学夏洛特分校）；U. of Central Florida（中佛罗里达大学）
We introduce a new formal model -- based on the mathematical construct of sheaves -- for representing contradictory information in textual sources. This model has the advantage of letting us (a) identify the causes of the inconsistency; (b) measure how strong it is; (c) and do something about it, e.g. suggest ways to reconcile inconsistent advice. This model naturally represents the distinction between contradictions and disagreements. It is based on the idea of representing natural language sentences as formulas with parameters sitting on lattices, creating partial orders based on predicates …
Disagreement as Data: Reasoning Trace Analytics in Multi-Agent Systems
University at Albany（纽约州立大学阿尔巴尼分校）；Carnegie Mellon University（卡内基梅隆大学）；Arizona State University（亚利桑那州立大学）；Le Mans University（勒曼大学）；University of California, Irvine（加州大学尔湾分校）；Indian Institute of Technology Bombay（印度班加罗尔理工学院）；Extuitive Inc. (Flagship Pioneering)（Extuitive公司（Flagship Pioneering））
Lack of labeled data is a main obstacle in relation extraction. Semi-supervised relation extraction (SSRE) has been proven to be a promising way for this problem through annotating unlabeled samples as additional training data. Almost all prior researches along this line adopt multiple models to make the annotations more reliable by taking the intersection set of predicted results from these models. However, the difference set, which contains rich information about unlabeled data, has been long neglected by prior studies. In this paper, we propose to learn not only from the consensus but also …
Training-Free Cultural Alignment of Large Language Models via Persona Disagreement
Faculty of Information and Technology, University of Science, Vietnam National University（信息与技术学院，科学大学，越南国家大学）；Department of Computer Science, University of Warwick（计算机科学系，沃里克大学）；School of Computing, Engineering and Digital Technologies, Teesside University（计算、工程和数字技术学院，泰赛德大学）
When Reviews Disagree: Fine-Grained Contradiction Analysis in Scientific Peer Reviews
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India（印度理工学院帕纳瓦分校计算机科学与工程系）；School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India（比哈尔邦布尔萨大学计算机工程学院）
Auditing medical multi-agent AI reveals risks of false consensus
National Engineering Research Center for Software Engineering, Peking University（北京大学软件工程国家工程研究中心）；School of Computing and Data Science, The University of Hong Kong（香港大学计算机与数据科学学院）；Department of Nephrology, Peking University Third Hospital（北京大学第三医院肾内科）；Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Department of Lymphoma, Peking University Cancer Hospital & Institute（教育部癌症发生与转化研究重点实验室、北京大学肿瘤医院淋巴瘤科）；Department of Automation, Tsinghua University（清华大学自动化系）；Centre for Medical Informatics, The University of Edinburgh（爱丁堡大学医学信息学中心）；Health Data Research UK（英国健康数据研究机构）；Lee Kong Chian School of Medicine, Nanyang Technological University（南洋理工大学李科贤医学院）；State Key Laboratory for Novel Software Technology, School of Computer Science, Nanjing University（南京大学新型软件技术国家重点实验室、计算机科学学院）
The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation
University of Toronto Computer Science（多伦多大学计算机科学系）；University of Toronto Engineering Science（多伦多大学工程科学系）；Carnegie Mellon University School of Computer Science（卡内基梅隆大学计算机科学学院）；Yale University American Studies（耶鲁大学美国研究系）；University of Waterloo Computer Science（滑铁卢大学计算机科学系）；Google Research（谷歌研究）；University of Toronto（多伦多大学）；Carnegie Mellon University（卡内基梅隆大学）；Yale University（耶鲁大学）；University of Waterloo（滑铁卢大学）
