大模型推理能力

GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling

Siqi Li, Yufan Shen, Xiangnan Chen, Jiayi Chen, Hengwei Ju, Haodong Duan, Song Mao, Hongbin Zhou, Bo Zhang, Bin Fu, Pinlong Cai, Licheng Wen, Botian Shi, Yong Liu, Xinyu Cai, Yu Qiao

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Zhejiang University（浙江大学） ； School of Science and Engineering, The Chinese University of Hong Kong（香港中文大学科学与工程学院） ； Fudan University（复旦大学） ； Shanghai Innovation Institute（上海创新研究院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association

Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song

机构 * Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China（香港科技大学计算机科学与工程系） ； Amazon.com Inc（亚马逊公司） ； Department of Computer Science & Engineering, Texas A&M University（德克萨斯农工大学计算机科学与工程系） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL

Comments ACL2025

Think Only When You Need with Large Hybrid-Reasoning Models

Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, Furu Wei

机构 * Microsoft Research（微软研究院） ； Peking University（北京大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models

Guanghao Zhou, Panjia Qiu, Cen Chen, Jie Wang, Zheming Yang, Jian Xu, Minghui Qiu

机构 * East China Normal University（东华师范大学） ； ByteDance（字节跳动） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset

Weiqi Wang, Yangqiu Song

机构 * Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China（计算机科学与工程系，香港科技大学，香港特别行政区，中国） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments ACL2025

MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports

Kevin Wu, Eric Wu, Rahul Thapa, Kevin Wei, Angela Zhang, Arvind Suresh, Jacqueline J. Tao, Min Woo Sun, Alejandro Lozano, James Zou

机构 * Stanford University（斯坦福大学） ； University of Southern California（南加州大学） ； University of California, San Francisco（旧金山大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Computational Reasoning of Large Language Models

Haitao Wu, Zongbo Han, Joey Tianyi Zhou, Huaxi Huang, Changqing Zhang

机构 * Tianjin University（天津大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； A ∗ STAR Centre for Frontier AI Research（A STAR前沿人工智能研究中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models

Shi Qiu, Shaoyang Guo, Zhuo-Yang Song, Yunbo Sun, Zeyu Cai, Jiashen Wei, Tianyu Luo, Yixuan Yin, Haoxu Zhang, Yi Hu, Chenyang Wang, Chencheng Tang, Haoling Chang, Qi Liu, Ziheng Zhou, Tianyu Zhang, Jingtian Zhang, Zhangyi Liu, Minghao Li, Yuku Zhang, Boxuan Jing, Xianqi Yin, Yutong Ren, Zizhuo Fu, Jiaming Ji, Weike Wang, Xudong Tian, Anqi Lv, Laifu Man, Jianxiang Li, Feiyu Tao, Qihua Sun, Zhou Liang, Yushu Mu, Zhongxuan Li, Jing-Jun Zhang, Shutao Zhang, Xiaotian Li, Xingqi Xia, Jiawei Lin, Zheyu Shen, Jiahang Chen, Qiuhao Xiong, Binran Wang, Fengyuan Wang, Ziyang Ni, Bohan Zhang, Fan Cui, Changkun Shao, Qing-Hong Cao, Ming-xing Luo, Yaodong Yang, Muhan Zhang, Hua Xing Zhu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 34 pages ,12 figures, 7 tables, latest update in 2025/05/18

SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarios

Weidong Zhan, Yue Wang, Nan Hu, Liming Xiao, Jingyuan Ma, Yuhang Qin, Zheng Li, Yixin Yang, Sirui Deng, Jinkun Ding, Wenhan Ma, Rui Li, Weilin Luo, Qun Liu, Zhifang Sui

机构 * Center for Chinese Linguistics, Department of Chinese Language and Literature, Peking University（北京大学中文语言文学系中文语言学中心） ； School of Computer Science, State Key Laboratory of Multimedia Information Processing, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室） ； Huawei Noah’s Ark Lab, China（华为诺亚实验室，中国） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution

Ruiyang Xu, Jialun Cao, Yaojie Lu, Ming Wen, Hongyu Lin, Xianpei Han, Ben He, Shing-Chi Cheung, Le Sun

机构 * Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所信息处理实验室） ； University of Chinese Academy of Sciences（中国科学院大学） ； The Hong Kong University of Science and Technology（香港科技大学） ； Huazhong University of Science and Technology（华中科技大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 18pages, Accepted to ACL 2025 Main Conference

Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token Level Granularity

Chan-Jan Hsu, Davide Buffelli, Jamie McGowan, Feng-Ting Liao, Yi-Chang Chen, Sattar Vakili, Da-shan Shiu

机构 * MediaTek Research（联发科研究） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training

Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Yunjie Ji, Han Zhao, Xiangang Li

机构 * a-m-team（a-m团队） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

R^3-VQA: "Read the Room" by Video Social Reasoning

Lixing Niu, Jiapeng Li, Xingping Yu, Shu Wang, Ruining Feng, Bo Wu, Ping Wei, Yisen Wang, Lifeng Fan

机构 * School of Intelligence Science and Technology, Peking University（北京理工大学智能科学与技术学院） ； College of Artificial Intelligence, Xi’an Jiaotong University（西安交通大学人工智能学院） ； State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence（一般人工智能国家重点实验室） ； Yuanpei College, Peking University（北京大学元培学院） ； Tsinghua University（清华大学） ； University of California, Los Angeles（加州大学洛杉矶分校） ； MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Exploring the Potential of Offline RL for Reasoning in LLMs: A Preliminary Study

Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Yunjie Ji, Han Zhao, Xiangang Li

机构 * a-m-team（a-m团队） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

Jen-Hao Cheng, Vivian Wang, Huayu Wang, Huapeng Zhou, Yi-Hao Peng, Hou-I Liu, Hsiang-Wei Huang, Kuang-Ming Chen, Cheng-Yen Yang, Wenhao Chai, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang

机构 * University of Washington（华盛顿大学） ； Carnegie Mellon University（卡内基梅隆大学） ； National Yang Ming Chiao Tung University ； Microsoft（微软公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model

Haiyan Qin, Zhiwei Xie, Jingjing Li, Liangchen Li, Xiaotong Feng, Junzhan Liu, Wang Kang

机构 * National Key Laboratory of Spintronics, Hangzhou International Innovation Institute（国家磁电子重点实验室、杭州国际创新研究院） ； School of Integrated Circuit Science and Engineering, Beihang University（集成电路科学与工程学院、北京航空航天大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 9 pages, 4 figures

SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning

Cheng Wen, Tingwei Guo, Shuaijiang Zhao, Wei Zou, Xiangang Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs

Huichi Zhou, Zehao Xu, Munan Zhao, Kaihong Li, Yiqiang Li, Hongtao Wang

机构 * Imperial College London（伦敦帝国学院） ； North China Electric Power University（华北电力大学） ； Sun Yat-sen University（中山大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 5 pages, 2 figures

CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges

Yu Li, Qizhi Pei, Mengyuan Sun, Honglin Lin, Chenlin Ming, Xin Gao, Jiang Wu, Conghui He, Lijun Wu

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Wuhan University（武汉大学） ； Renmin University of China（中国人民大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Work in progress

MASR: Self-Reflective Reasoning through Multimodal Hierarchical Attention Focusing for Agent-based Video Understanding

Shiwen Cao, Zhaoxing Zhang, Junming Jiao, Juyi Qiao, Guowen Song, Rong Shen, Xiangbing Meng

机构 * Li Auto Inc.（利亚汽车公司） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models

Hongyu Wang, Jiayu Xu, Senwei Xie, Ruiping Wang, Jialin Li, Zhaojie Xie, Bin Zhang, Chuyan Xiong, Xilin Chen

机构 * Key Laboratory of AI Safety of Chinese Academy of Sciences (CAS)（中国科学院人工智能安全重点实验室（中国科学院）） ； Institute of Computing Technology, CAS（中国科学院计算技术研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Work in progress

GeoSense: Evaluating Identification and Application of Geometric Principles in Multimodal Reasoning

Liangyu Xu, Yingxiu Zhao, Jingyun Wang, Yingyao Wang, Bu Pi, Chen Wang, Mingliang Zhang, Jihao Gu, Xiang Li, Xiaoyong Zhu, Jun Song, Bo Zheng

机构 * Taobao & Tmall Group of Alibaba Beijing China（阿里巴巴北京公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 10 pages, 8 figures

Impact of Noise on LLM-Models Performance in Abstraction and Reasoning Corpus (ARC) Tasks with Model Temperature Considerations

Nikhil Khandalkar, Pavan Yadav, Krishna Shinde, Lokesh B. Ramegowda, Rajarshi Das

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 60 pages, 25 figures

Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

Di Zhang, Junxian Li, Jingdi Lei, Xunzhi Wang, Yujie Liu, Zonglin Yang, Jiatong Li, Weida Wang, Suorong Yang, Jianbo Wu, Peng Ye, Wanli Ouyang, Dongzhan Zhou

机构 * Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Shanghai Jiaotong University（上海交通大学） ； Nankai University（南开大学） ； Shanghai University（上海大学） ； Nanyang Technological University（南洋理工大学） ； Hong Kong Polytechnic University（香港理工大学） ； Tongji University（同济大学） ； Nanjing University（南京大学） ； University of California, Merced（加州大学梅德福分校） ； Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 16 pages, 11 figures

DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large Language Models

Jie Zhu, Qian Chen, Huaixia Dou, Junhui Li, Lifan Guo, Feng Chen, Chi Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning in Multimodal LLMs

David Ma, Yuanxing Zhang, Jincheng Ren, Jarvis Guo, Yifan Yao, Zhenlin Wei, Zhenzhu Yang, Zhongyuan Peng, Boyu Feng, Jun Ma, Xiao Gu, Zhoufutu Wen, King Zhu, Yancheng He, Meng Cao, Shiwen Ni, Jiaheng Liu, Wenhao Huang, Ge Zhang, Xiaojie Jin

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

CoLoTa: A Dataset for Entity-based Commonsense Reasoning over Long-Tail Knowledge

Armin Toroghi, Willis Guo, Scott Sanner

机构 * University of Toronto（多伦多大学） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videos

Rui Chen, Lei Sun, Jing Tang, Geng Li, Xiangxiang Chu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 10 pages, 4 figures

What the HellaSwag? On the Validity of Common-Sense Reasoning Benchmarks

Pavel Chizhov, Mattia Nee, Pierre-Carl Langlais, Ivan P. Yamshchikov

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

Pengfei Zhou, Fanrui Zhang, Xiaopeng Peng, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 11 pages, 8 figures