大模型推理能力

Dissecting Clinical Reasoning in Language Models: A Comparative Study of Prompts and Model Adaptation Strategies

Mael Jullien, Marco Valentino, Leonardo Ranaldi, Andre Freitas

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Nunchi-Bench: Benchmarking Language Models on Cultural Reasoning with a Focus on Korean Superstition

Kyuhee Kim, Sangah Lee

机构 * EPFL Lausanne, Switzerland（洛桑联邦理工学院） ； Seoul National University（首尔国立大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

KERAP: A Knowledge-Enhanced Reasoning Approach for Accurate Zero-shot Diagnosis Prediction Using Multi-agent LLMs

Yuzhang Xie, Hejie Cui, Ziyang Zhang, Jiaying Lu, Kai Shu, Fadi Nahab, Xiao Hu, Carl Yang

机构 * Emory University（埃默里大学） ； Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Journal ref American Medical Informatics Association (AMIA) 2025 Annual Symposium, Oral

Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs

Akram Mustafa, Usman Naseem, Mostafa Rahimi Azghadi

机构 * College of Science and Engineering, James Cook University（科学与工程学院，詹姆斯库克大学） ； School of Computing, Macquarie University（计算学院，麦考瑞大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness

Zixin Chen, Hongzhan Lin, Kaixin Li, Ziyang Luo, Zhen Ye, Guang Chen, Zhiyong Huang, Jing Ma

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2025

Two-Stage Reasoning-Infused Learning: Improving Classification with LLM-Generated Reasoning

Mads Henrichsen, Rasmus Krebs

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models

Bowen Wang, Jiuyang Chang, Yiming Qian, Guoxin Chen, Junhao Chen, Zhouqiang Jiang, Jiahao Zhang, Yuta Nakashima, Hajime Nagahara

机构 * Premium Research Institute for Human Metaverse Medicine (WPI-PRIMe)（人类元宇宙医学高级研究机构（WPI-PRIMe）） ； Osaka University（大阪大学） ； Department of Cardiology, The First Affiliated Hospital of Dalian Medical University（大连医科大学第一附属医院心内科） ； Institute of Computing Technology, Chinese Academy of Science（中国科学院计算技术研究所） ； D3 Center, Osaka University（大阪大学D3中心） ； Agency for Science, Technology and Research (A*STAR)（科技研究局（A*STAR）） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2024 D&B Track

DABstep: Data Agent Benchmark for Multi-step Reasoning

Alex Egg, Martin Iglesias Goyanes, Friso Kingma, Andreu Mora, Leandro von Werra, Thomas Wolf

机构 * Adyen（阿德耶恩） ； Hugging Face 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 13 pages, 5 figures

MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI

Huanjin Yao, Jiaxing Huang, Yawen Qiu, Michael K. Chen, Wenzheng Liu, Wei Zhang, Wenjie Zeng, Xikun Zhang, Jingyi Zhang, Yuxin Song, Wenhao Wu, Dacheng Tao

机构 * Nanyang Technological University（南洋理工大学） ； Tsinghua University（清华大学） ； Baidu Inc.（百度公司） ； University of California（加州大学） ； University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Technical report

MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot

Xuejiao Zhao, Siyan Liu, Su-Yin Yang, Chunyan Miao

机构 * LILY Research Centre（LILY研究机构） ； Nanyang Technological University（南洋理工大学） ； Tan Tock Seng Hospital（坦托克森医院） ； Woodlands Health（伍尔德斯健康） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind

Andrei Lupu, Timon Willi, Jakob Foerster

机构 * FAIR at Meta（Meta 的 FAIR） ； University of Oxford（牛津大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 41 pages, 19 figures

TRAIL: Trace Reasoning and Agentic Issue Localization

Darshan Deshpande, Varun Gangal, Hersh Mehta, Jitin Krishnan, Anand Kannappan, Rebecca Qian

机构 * Patronus AI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Dataset: https://huggingface.co/datasets/PatronusAI/TRAIL

ReasonGRM: Enhancing Generative Reward Models through Large Reasoning Models

Bin Chen, Xinzge Gao, Chuanrui Hu, Penghang Yu, Hua Zhang, Bing-Kun Bao

机构 * Qihoo360（奇安信） ； Nanjing University of Posts and Telecommunications（南京邮电大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Under the Shadow of Babel: How Language Shapes Reasoning in LLMs

Chenxi Wang, Yixuan Zhang, Lang Gao, Zixiang Xu, Zirui Song, Yanbo Wang, Xiuying Chen

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 10 figures

GraphRAG-Bench: Challenging Domain-Specific Reasoning for Evaluating Graph Retrieval-Augmented Generation

Yilin Xiao, Junnan Dong, Chuang Zhou, Su Dong, Qian-wen Zhang, Di Yin, Xing Sun, Xiao Huang

机构 * The Hong Kong Polytechnic University（香港理工大学） ； Tencent Youtu Lab（腾讯优图实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

On Path to Multimodal Historical Reasoning: HistBench and HistAgent

Jiahao Qiu, Fulian Xiao, Yimin Wang, Yuchen Mao, Yijia Chen, Xinzhe Juan, Shu Zhang, Siran Wang, Xuan Qi, Tongcheng Zhang, Zixin Yao, Jiacheng Guo, Yifu Lu, Charles Argon, Jundi Cui, Daixin Chen, Junran Zhou, Shuyao Zhou, Zhanpeng Zhou, Ling Yang, Shilong Liu, Hongru Wang, Kaixuan Huang, Xun Jiang, Yuming Cao, Yue Chen, Yunfei Chen, Zhengyi Chen, Ruowei Dai, Mengqiu Deng, Jiye Fu, Yunting Gu, Zijie Guan, Zirui Huang, Xiaoyan Ji, Yumeng Jiang, Delong Kong, Haolong Li, Jiaqi Li, Ruipeng Li, Tianze Li, Zhuoran Li, Haixia Lian, Mengyue Lin, Xudong Liu, Jiayi Lu, Jinghan Lu, Wanyu Luo, Ziyue Luo, Zihao Pu, Zhi Qiao, Ruihuan Ren, Liang Wan, Ruixiang Wang, Tianhui Wang, Yang Wang, Zeyu Wang, Zihua Wang, Yujia Wu, Zhaoyi Wu, Hao Xin, Weiao Xing, Ruojun Xiong, Weijie Xu, Yao Shu, Yao Xiao, Xiaorui Yang, Yuchen Yang, Nan Yi, Jiadong Yu, Yangyuxuan Yu, Huiting Zeng, Danni Zhang, Yunjie Zhang, Zhaoyu Zhang, Zhiheng Zhang, Xiaofeng Zheng, Peirong Zhou, Linyan Zhong, Xiaoyin Zong, Ying Zhao, Zhenxin Chen, Lin Ding, Xiaoyu Gao, Bingbing Gong, Yichao Li, Yang Liao, Guang Ma, Tianyuan Ma, Xinrui Sun, Tianyi Wang, Han Xia, Ruobing Xian, Gen Ye, Tengfei Yu, Wentao Zhang, Yuxi Wang, Xi Gao, Mengdi Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 7 figures

RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation

Xinnuo Xu, Rachel Lawrence, Kshitij Dubey, Atharva Pandey, Risa Ueno, Fabian Falck, Aditya V. Nori, Rahul Sharma, Amit Sharma, Javier Gonzalez

机构 * Microsoft Research Cambridge, UK（微软研究院剑桥分部） ； Microsoft Research India, India（微软研究院印度分部） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ICML 2025

MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance

Joseph J. Peper, Wenzhao Qiu, Ali Payani, Lu Wang

机构 * Computer Science and Engineering, University of Michigan（计算机科学与工程系，密歇根大学） ； Cisco Research（思科研究） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Findings

Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs

Ling Team, Bin Hu, Cai Chen, Deng Zhao, Ding Liu, Dingnan Jin, Feng Zhu, Hao Dai, Hongzhi Luan, Jia Guo, Jiaming Liu, Jiewei Wu, Jun Mei, Jun Zhou, Junbo Zhao, Junwu Xiong, Kaihong Zhang, Kuan Xu, Lei Liang, Liang Jiang, Liangcheng Fu, Longfei Zheng, Qiang Gao, Qing Cui, Quan Wan, Shaomian Zheng, Shuaicheng Li, Tongkai Yang, Wang Ren, Xiaodong Yan, Xiaopei Wan, Xiaoyun Feng, Xin Zhao, Xinxing Yang, Xinyu Kong, Xuemin Yang, Yang Li, Yingting Wu, Yongkang Liu, Zhankai Xu, Zhenduo Zhang, Zhenglei Zhou, Zhenyu Huang, Zhiqiang Zhang, Zihao Wang, Zujie Wen

机构 * Ling Team Inclusion AI（ling团队Inclusion AI） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Technical Report

MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?

Xixian Yong, Jianxun Lian, Xiaoyuan Yi, Xiao Zhou, Xing Xie

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning

Can Polat, Hasan Kurban, Erchin Serpedin, Mustafa Kurban

机构 * Dept. of Electrical and Computer Engineering, Texas A&M University（电气与计算机工程系，德克萨斯A&M大学） ； College of Science and Engineering, Hamad Bin Khalifa University（科学与工程学院，哈马德·本·卡西姆大学） ； Dept. of Electrical and Computer Engineering, Texas A&M University at Qatar（电气与计算机工程系，德克萨斯A&M大学（卡塔尔）） ； Dept. of Prosthetics and Orthotics, Ankara University（假肢与矫形系，安卡拉大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions

Siyin Wang, Wenyi Yu, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Lu Lu, Yu Tsao, Junichi Yamagishi, Yuxuan Wang, Chao Zhang

机构 * Tsinghua University（清华大学） ； ByteDance（字节跳动） ； Academia Sinica（中国科学院） ； National Institute of Informatics（日本信息处理研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 10 figures

A dataset of questions on decision-theoretic reasoning in Newcomb-like problems

Caspar Oesterheld, Emery Cooper, Miles Kodama, Linh Chi Nguyen, Ethan Perez

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 48 pages, 15 figures; code and data at https://github.com/casparoe/newcomblike_questions_dataset

Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning

LASA Team, Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen, Chaojun Wang, Jie Tan, Deli Zhao, Tingyang Xu, Hao Zhang, Yu Rong

机构 * DAMO Academy, Alibaba Group（达摩院，阿里巴巴集团） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Technical Report, 53 pages, 25 tables, and 16 figures. Our webpage is https://alibaba-damo-academy.github.io/lingshu/

SlotPi: Physics-informed Object-centric Reasoning Models

Jian Li, Wan Han, Ning Lin, Yu-Liang Zhan, Ruizhi Chengze, Haining Wang, Yi Zhang, Hongsheng Liu, Zidong Wang, Fan Yu, Hao Sun

机构 * Renmin University of China（中国人民大学） ； Huawei Technologies Ltd.（华为技术有限公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

CHANCERY: Evaluating Corporate Governance Reasoning Capabilities in Language Models

Lucas Irwin, Arda Kaz, Peiyao Sheng, Sewoong Oh, Pramod Viswanath

机构 * Sentient Labs（Sentient实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Models

Danae Sánchez Villegas, Ingo Ziegler, Desmond Elliott

机构 * University of Copenhagen（哥本哈根大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments Code, dataset, and checkpoints are publicly available at https://github.com/danaesavi/ImageChain; v2: added human annotation study to validate SimRate

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning

Yizhao Gao, Shuming Guo, Shijie Cao, Yuqing Xia, Yu Cheng, Lei Wang, Lingxiao Ma, Yutao Sun, Tianzhu Ye, Li Dong, Hayden Kwok-Hay So, Yu Hua, Ting Cao, Fan Yang, Mao Yang

机构 * Microsoft Research（微软研究院） ； The University of Hong Kong（香港大学） ； Huazhong University of Science and Technology（华中科技大学） ； Peking University（北京大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim $\rightarrow$ Evidence Reasoning

Shashidhar Reddy Javaji, Yupeng Cao, Haohang Li, Yangyang Yu, Nikhil Muralidhar, Zining Zhu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 6 figures, Under review

Interpretable and Reliable Detection of AI-Generated Images via Grounded Reasoning in MLLMs

Yikun Ji, Hong Yan, Jun Lan, Huijia Zhu, Weiqiang Wang, Qi Fan, Liqing Zhang, Jianfu Zhang

机构 * Shanghai Jiao Tong University（上海交通大学） ； Ant Group（蚂蚁集团） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI