大模型推理能力

Doing Good or Doing Right? Exploring the Weakness of Commonsense Causal Reasoning Models

Mingyue Han, Yinglin Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments ACL2021, Main Conference, Short Paper

Journal ref Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Short Papers), pages 151-157, 2021

MedTVT-R1: A Multimodal LLM Empowering Medical Reasoning and Diagnosis

Yuting Zhang, Kaishen Yuan, Hao Lu, Yutao Yue, Jintai Chen, Kaishun Wu

机构 * The Hong Kong University of Science & Technology (Guangzhou)（香港科技大学（广州）） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Standard Applicability Judgment and Cross-jurisdictional Reasoning: A RAG-based Framework for Medical Device Compliance

Yu Han, Aaron Ceross, Jeroen H. M. Bergmann

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts

Abhay Gupta, Michael Lu, Kevin Zhu, Sean O'Brien, Vasu Sharma

机构 * Algoverse AI Research（Algoverse AI研究机构） ； University of California, Berkeley（加州大学伯克利分校） ； Meta FAIR Lab（Meta FAIR实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

RadFabric: Agentic AI System with Reasoning Capability for Radiology

Wenting Chen, Yi Dong, Zhaojun Ding, Yucheng Shi, Yifan Zhou, Fang Zeng, Yijun Luo, Tianyu Lin, Yihang Su, Yichen Wu, Kai Zhang, Zhen Xiang, Tianming Liu, Ninghao Liu, Lichao Sun, Yixuan Yuan, Xiang Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 4 figures, 2 tables

CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs

Ai Jian, Weijie Qiu, Xiaokun Wang, Peiyu Wang, Yunzhuo Hao, Jiangbo Pei, Yichen Wei, Yi Peng, Xuchen Song

机构 * Skywork AI ； Kunlun Inc. 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 36 pages

OneEval: Benchmarking LLM Knowledge-intensive Reasoning over Diverse Knowledge Bases

Yongrui Chen, Zhiqiang Liu, Jing Yu, Lin Ren, Nan Hu, Xinbang Dai, Jiajun Liu, Jiazhen Kang, Shenyu Zhang, Xinda Wang, Keyan Ding, Pengfei Shen, Haolei Zhu, Hongjie Deng, Yisong Wang, Tongtong Wu, Sheng Bi, Wen Zhang, Tianxing Wu, Qiu Ji, Haofen Wang, Wenliang Chen, Huajun Chen, Guilin Qi

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks

Sanjoy Chowdhury, Mohamed Elmoghany, Yohan Abeysinghe, Junjie Fei, Sayan Nag, Salman Khan, Mohamed Elhoseiny, Dinesh Manocha

机构 * University of Maryland, College Park（马里兰大学College Park分校） ； KAUST（卡尔斯兰大学） ； MBZUAI（马克斯·普朗克人工智能研究所） ； University of Toronto（多伦多大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Audio-visual learning, Audio-Visual RAG, Multi-Video Linkage

MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space

Anshul Singh, Chris Biemann, Jan Strich

机构 * Department of Information Technology, Panjab University（信息技术系，旁遮普大学） ； Language Technology Group, Universität Hamburg（语言技术组，汉堡大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

OIBench: Benchmarking Strong Reasoning Models with Olympiad in Informatics

Yaoming Zhu, Junxin Wang, Yiyang Li, Lin Qiu, ZongYu Wang, Jun Xu, Xuezhi Cao, Yuhuai Wei, Mingshi Wang, Xunliang Cai, Rong Ma

机构 * AGI-Eval ； Beijing Normal University（北京师范大学） ； Meituan（美团） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

WGSR-Bench: Wargame-based Game-theoretic Strategic Reasoning Benchmark for Large Language Models

Qiyue Yin, Pei Xu, Qiaozhe Li, Shengda Liu, Shengqi Shen, Tong Wang, Yihong Han, Xiaonan Zhao, Likun Yang, Shiyue Cao, Shiyu Qiu, Yuxuan Liu, Shizhao Yu, Lei Cui, Chengxin Yan, Jie Sun, Xiangquan Tang, Kaiqi Huang

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 15 pages, 17 figures

Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy

Sushant Gautam, Michael A. Riegler, Pål Halvorsen

机构 * Simula Metropolitan Center for Digital Engineering (SimulaMet)（Simula Metropolitan Center for Digital Engineering） ； Oslo Metropolitan University (OsloMet)（Oslo Metropolitan University） ； Simula Research Laboratory 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts

Ruiran Su, Jiasheng Si, Zhijiang Guo, Janet B. Pierrehumbert

机构 * University of Oxford（牛津大学） ； Qilu University of Technology(Shandong Academy of Sciences)（齐鲁工业大学（山东科学院）） ； Hong Kong University of Science and Technology（香港科学大学） ； Hong Kong University of Science and Technology (Guangzhou)（香港科学大学（广州）） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

HAIBU-ReMUD: Reasoning Multimodal Ultrasound Dataset and Model Bridging to General Specific Domains

Shijie Wang, Yilun Zhang, Zeyu Lai, Dexing Kong

机构 * School of Mathematical Sciences, Zhejiang University（浙江大学数学科学学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Reasoning Multimodal Large Language Model: Data Contamination and Dynamic Evaluation

Ming Liu, Wensheng Zhang

机构 * Department of Computer Science（计算机科学系） ； Iowa State University（爱荷华州立大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

BioHopR: A Benchmark for Multi-Hop, Multi-Answer Reasoning in Biomedical Domain

Yunsoo Kim, Yusuf Abdulle, Honghan Wu

机构 * UCL（伦敦大学学院） ； King’s College London（伦敦国王学院） ； University of Glasgow（格拉斯哥大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Towards Foundation Model on Temporal Knowledge Graph Reasoning

Jiaxin Pan, Mojtaba Nayyeri, Osama Mohammed, Daniel Hernandez, Rongchuan Zhang, Cheng Cheng, Steffen Staab

机构 * University of Stuttgart（斯图加特大学） ； University of Southampton（南安普顿大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Understanding Financial Reasoning in AI: A Multimodal Benchmark and Error Learning Approach

Shuangyan Deng, Haizhou Peng, Jiachen Xu, Chunhou Liu, Ciprian Doru Giurcuaneanu, Jiamou Liu

机构 * University of Auckland（奥克兰大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning

Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Wei Ma, Lyuye Zhang, Yang Liu, Yingjiu Li

机构 * Nanyang Technological University, Singapore（南洋理工大学） ； The Hong Kong University of Science（香港科学与技术大学） ； MetaTrust Labs（MetaTrust实验室） ； Singapore Management University, Singapore（新加坡管理学院） ； University of Oregon, Eugene, OR, USA（俄勒冈大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments This is a technical report by Nanyang Technological University. Updated to support Solidity, Java and C/C++

BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactions

Saptarshi Sengupta, Shuhua Yang, Paul Kwong Yu, Fali Wang, Suhang Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests

Shiyi Xu, Yiwen Hu, Yingqian Min, Zhipeng Chen, Wayne Xin Zhao, Ji-Rong Wen

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents

Juhyun Oh, Eunsu Kim, Alice Oh

机构 * School of Computing（计算机学院） ； KAIST（韩国科学技术院） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL

OpenThoughts: Data Recipes for Reasoning Models

Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, Ludwig Schmidt

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments https://www.openthoughts.ai/blog/ot3. arXiv admin note: text overlap with arXiv:2505.23754 by other authors

MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models

Huanqia Cai, Yijun Yang, Winston Hu

机构 * Tencent Hunyuan Team（腾讯文言团队） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learning

Ming Yin, Yuanhao Qu, Ling Yang, Le Cong, Mengdi Wang

机构 * Princeton University（普林斯顿大学） ； Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning

Peiqi Sui, Juan Diego Rodriguez, Philippe Laban, Dean Murphy, Joseph P. Dexter, Richard Jean So, Samuel Baker, Pramit Chaudhuri

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments ACL 2025 main

Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning

Fangyu Lei, Jinxiang Meng, Yiming Huang, Tinghong Chen, Yun Zhang, Shizhu He, Jun Zhao, Kang Liu

机构 * Institute of Automation, CAS（中国科学院自动化研究所） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Work in progress

Deep Temporal Reasoning in Video Language Models: A Cross-Linguistic Evaluation of Action Duration and Completion through Perfect Times

Olga Loginova, Sofía Ortega Loguinova

机构 * University of Trento（特伦托大学） ； Maastricht University（马斯特里赫特大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations

Pardis Sadat Zahraei, Ali Emami

机构 * Independent Researcher（独立研究者） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Accepted to Findings of ACL 2025

SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues

Martin Kuo, Jianyi Zhang, Aolin Ding, Louis DiValentin, Amin Hass, Benjamin F Morris, Isaac Jacobson, Randolph Linderman, James Kiessling, Nicolas Ramos, Bhavna Gopal, Maziyar Baran Pouyan, Changwei Liu, Hai Li, Yiran Chen

机构 * Center for Computational Evolutionary Intelligence, Duke University（计算进化智能中心，杜克大学） ； Accenture, USA（Accenture美国公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL