The task of abductive natural language inference (αnli), to decide which hypothesis is the more likely explanation for a set of observations, is a particularly difficult type of NLI. Instead of just determining a causal relationship, it requires common sense to also evaluate how reasonable an explanation is. All recent competitive systems build on top of contextualized representations and make use of transformer architectures for learning an NLI model. When somebody is faced with a particular NLI task, they need to select the best model that is available. This is a time-consuming and resource-intense endeavour. To solve this practical problem, we propose a simple method for predicting the performance without actually fine-tuning the model. We do this by testing how well the pre-trained models perform on the αnli task when just comparing sentence embeddings with cosine similarity to what the performance that is achieved when training a classifier on top of these embeddings. We show that the accuracy of the cosine similarity approach correlates strongly with the accuracy of the classification approach with a Pearson correlation coefficient of 0.65. Since the similarity computation is orders of magnitude faster to compute on a given dataset (less than a minute vs. hours), our method can lead to significant time savings in the process of model selection.
Inferring Latent Intentions: Attributional Natural Language Inference in LLM Agents
University of Manchester(曼彻斯特大学);University of Sheffield(谢菲尔德大学);Idiap Research Institute(Idiap研究 institute);National Biomarker Centre, CRUK-MI, University of Manchester(国家生物标志物中心,CRUK-MI,曼彻斯特大学)
Allen Institute for AI(艾伦人工智能研究院);Facebook AI(脸书人工智能研究院);Paul G. Allen School of Computer Science & Engineering(保罗·G·艾伦计算机科学与工程学院)
溯因推理是对最合理解释的推理。例如,如果珍妮下班回家后发现家里一片狼借,她记得自己开着一扇窗户,她可以假设一个小偷闯入了她的房子,并造成了混乱,这是最合理的解释。虽然溯因一直被认为是人们如何解释和阅读自然语言字里行间的核心(Hobbs et al.,1988年),在支持溯因自然语言推理和生成方面的研究相对较少。我们提出的第一项研究,调查基于语言的溯因推理的可行性。我们引入了一个挑战数据集ART,它由超过20 k个常识性的叙事上下文和200 k个解释组成。基于这个数据集,我们概念化两个新的任务-(i)外展NLI:一个多项选择问答任务,选择更可能的解释,和(ii)外展NLG:一个条件生成任务,解释给定的观察自然语言。在外展NLI上,最佳模型的准确率为68.9%,远低于人类的91.4%。在外展NLG上,目前最好的语言生成器甚至更加挣扎,因为它们缺乏对人类来说微不足道的推理能力。我们的分析导致了对深度预训练的语言模型未能执行的推理类型的新见解-尽管它们在相关但定义更窄的NLI任务上表现出色-为未来的研究指明了有趣的途径。
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning
Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);University of California, Berkeley(加州大学伯克利分校);Boston University(波士顿大学);MIT-IBM Watson AI(麻省理工学院-IBM沃森人工智能实验室)
人类有非凡的能力进行溯因推理,并对图像的字面内容之外的内容进行假设。通过识别分散在场景中的具体视觉线索,我们几乎忍不住根据我们对世界的日常经验和知识,在文字场景之外做出可能的推断。例如,如果我们在路边看到“20英里/小时”的标志,我们可能会认为这条街位于住宅区(而不是高速公路上),即使没有房屋被拍到。机器能执行类似的视觉推理吗?我们提出了夏洛克,103 K图像的注释语料库,用于测试机器的溯因推理能力超出字面图像内容。我们采用自由观看范式:参与者首先观察并识别图像中的显著线索(例如,物体,动作),然后根据线索对场景进行合理的推断。总的来说,我们收集了363 K(线索,推理)对,形成了第一个溯因视觉推理数据集。使用我们的语料库,我们测试三个互补轴的溯因推理。我们评估模型的能力:i)从大型候选语料库中检索相关推断; ii)通过边界框定位推断的证据; iii)比较合理的推断,以匹配新收集的19 K Likert尺度判断的诊断语料库上的人类判断。虽然我们发现使用多任务目标微调CLIP-RN 50 x64的性能优于强大的基线,但模型性能和人类一致性之间存在显着的空间。数据、模型和排行榜可在http://visualabduction.com/获得
SemEval-2026 Task 12: Abductive Event Reasoning: Towards Real-World Event Causal Inference for Large Language Models
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China(认知与决策智能复杂系统重点实验室,自动化研究所,中国科学院,北京,中国);School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China(人工智能学院,中国科学院大学,北京,中国)
ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models
School of Informatics, Xiamen University, Xiamen, China(信息学院,厦门大学,厦门);School of Film, Xiamen University, Xiamen, China(电影学院,厦门大学,厦门);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism, Xiamen University, Xiamen, China(福建省和台湾非物质文化遗产数字保护与智能处理重点实验室,文化和旅游部,厦门大学,厦门);Institute of Artificial Intelligence, Xiamen University, Xiamen, China(人工智能学院,厦门大学,厦门);School of Computer and Data Science, Minjiang University, Fuzhou, China(计算机与数据科学学院,闽江大学,福州)
UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations
Cornell University(康奈尔大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学);University of Pittsburgh(匹兹堡大学);University of California, Berkeley(加州大学伯克利分校)
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Institute for Language, Cognition and Computation, University of Edinburgh(爱丁堡大学语言、认知与计算研究所);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)
CAS Key Lab of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所网络数据科学与技术重点实验室);University of Chinese Academy of Sciences(中国科学院大学)
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement
University of Manchester(曼彻斯特大学);Idiap Research Institute(Idiap研究所);National Biomarker Centre, CRUK-MI, University of Manchester(曼彻斯特大学英国癌症研究中心曼彻斯特研究所国家生物标志物中心)