大模型推理能力

On the Same Wavelength? Evaluating Pragmatic Reasoning in Language Models across Broad Concepts

Linlu Qiu, Cedegao E. Zhang, Joshua B. Tenenbaum, Yoon Kim, Roger P. Levy

机构 * MIT CSAIL（麻省理工学院计算机科学与人工智能实验室） ； MIT BCS（麻省理工学院脑科学与认知研究中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments EMNLP 2025 (Main)

Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards

Haoran He, Yuxiao Ye, Qingpeng Cai, Chen Hu, Binxing Jiao, Daxin Jiang, Ling Pan

机构 * Hong Kong University of Science and Technology（香港科学与技术大学） ； Kuaishou Technology（快手科技） ； StepFun 纠错

专题命中 推理评测 ：reasoning(title,abstract)；math reasoning(abstract)；分类 cs.AI、cs.LG

Comments 32 pages

GEAR: A General Evaluation Framework for Abductive Reasoning

Kaiyu He, Peilin Wu, Mian Zhang, Kun Wan, Wentian Zhao, Xinya Du, Zhiyu Chen

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Coda and Data: https://github.com/KaiyuHe998/GEAR-Abduction_evaluation

ADAM: A Diverse Archive of Mankind for Evaluating and Enhancing LLMs in Biographical Reasoning

Jasin Cekinmez, Omid Ghahroodi, Saad Fowad Chandle, Dhiman Gupta, Ehsaneddin Asgari

机构 * Qatar Computing Research Institute（卡塔尔计算研究所） ； Princeton University（普林斯顿大学） ； Virginia Tech（弗吉尼亚理工大学） ； Amity University（阿米蒂大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI、cs.LG

The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging

Xiaochong Lan, Yu Zheng, Shiteng Cao, Yong Li

机构 * Tsinghua University（清华大学） ； Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models

Jue Zhang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

机构 * Microsoft（微软） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP'25 (Main)

RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks

Amit Agarwal, Hitesh Laxmichand Patel, Srikant Panda, Hansa Meghwani, Jyotika Singh, Karan Dua, Paul Li, Tao Sheng, Sujith Ravi, Dan Roth

机构 * Oracle AI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in EMNLP 2025

Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； The Hong Kong University of Science and Technology（香港科技大学） ； Southeast University（东南大学） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2025 Main

KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning

Xilin Dang, Kexin Chen, Xiaorui Su, Ayush Noori, Iñaki Arango, Lucas Vittor, Xinyi Long, Yuyang Du, Marinka Zitnik, Pheng Ann Heng

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

MDD-Thinker: Towards Large Reasoning Models for Major Depressive Disorder Diagnosis

Yuyang Sha, Hongxin Pan, Gang Luo, Caijuan Shi, Jing Wang, Kefeng Li

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Context

Dasol Choi, Jungwhan Kim, Guijin Son

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Tagging the Thought: Unlocking Personalization Reasoning via Reinforcement Learning

Song Jin, Juntian Zhang, Yong Liu, Xun Zhang, Yufei Zhang, Fei Jiang, Guojun Yin, Wei Lin, Rui Yan

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院） ； Meituan（美团） ； Wuhan University（武汉大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking

Wuwei Zhang, Fangcong Yin, Howard Yen, Danqi Chen, Xi Ye

机构 * Princeton Language and Intelligence, Princeton University（普林斯顿语言与智能研究所，普林斯顿大学） ； The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments EMNLP 2025; Code at https://github.com/princeton-pli/QRHead

Truly Assessing Fluid Intelligence of Large Language Models through Dynamic Reasoning Evaluation

Yue Yang, MingKang Chen, Qihua Liu, Mengkang Hu, Qiguang Chen, Gengrui Zhang, Shuyue Hu, Guangtao Zhai, Yu Qiao, Yu Wang, Wenqi Shao, Ping Luo

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Harbin Institute of Technology（哈尔滨工业大学） ； The University of Hong Kong（香港大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

GRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs

Adrian-Marius Dumitran, Alexandra-Mihaela Danila, Angela-Liliana Dumitran

机构 * University of Bucharest Faculty of Mathematics and Computer Science（布加勒斯大学数学与计算机科学学院） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.LG

Comments Accepted as long paper @RANLP2025

Causally-Enhanced Reinforcement Policy Optimization

Xiangqi Wang, Yue Huang, Yujun Zhou, Xiaonan Luo, Kehan Guo, Xiangliang Zhang

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Reinforcement learning publication of 24 pages

VideoScore2: Think before You Score in Generative Video Evaluation

Xuan He, Dongfu Jiang, Ping Nie, Minghao Liu, Zhengxuan Jiang, Mingyi Su, Wentao Ma, Junru Lin, Chun Ye, Yi Lu, Keming Wu, Benjamin Schneider, Quy Duc Do, Zhuofeng Li, Yiming Jia, Yuxuan Zhang, Guo Cheng, Haozhe Wang, Wangchunshu Zhou, Qunshu Lin, Yuanxing Zhang, Ge Zhang, Wenhao Huang, Wenhu Chen

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； University of Waterloo（滑铁卢大学） ； Independent（独立） ； M-A-P ； University of Toronto（多伦多大学） ； Zhejiang University（浙江大学） ； Abaka AI 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing

Fuqing Bie, Shiyu Huang, Xijia Tao, Zhiqin Fang, Leyi Pan, Junzhe Chen, Min Ren, Liuyu Xiang, Zhaofeng He

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； XPENG ； The University of Hong Kong（香港大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation

Xi Ye, Fangcong Yin, Yinghui He, Joie Zhang, Howard Yen, Tianyu Gao, Greg Durrett, Danqi Chen

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments COLM 2025. Data and code available at: https://princeton-pli.github.io/LongProc

Preference-Based Long-Horizon Robotic Stacking with Multimodal Large Language Models

Wanming Yu, Adrian Röfer, Abhinav Valada, Sethu Vijayakumar

机构 * School of Informatics, University of Edinburgh（信息学院，爱丁堡大学） ； University of Freiburg（弗赖堡大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)

FindingDory: A Benchmark to Evaluate Memory in Embodied Agents

Karmesh Yadav, Yusuf Ali, Gunshi Gupta, Yarin Gal, Zsolt Kira

机构 * Georgia Tech（佐治亚理工学院） ； University of Oxford（牛津大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)

Comments Our dataset and code can be found at: https://findingdory-benchmark.github.io/

Patient-specific Biomolecular Instruction Tuning

Irsyad Adam, Zekai Chen, David Laub, Shaun Porwal, Arda Pekis, Kevin Brown

机构 * Standard Model Biomedicine（标准模型生物医学） ； University of California, San Diego（加州大学圣地亚哥分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Towards Better Generalization via Distributional Input Projection Network

Yifan Hao, Yanxin Lu, Hanning Zhang, Xinwei Shen, Tong Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation

Yuansheng Ni, Ping Nie, Kai Zou, Xiang Yue, Wenhu Chen

机构 * University of Waterloo（滑铁卢大学） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 推理评测 ：self-correction(abstract)；分类 cs.CL、cs.AI

Learning Regional Monsoon Patterns with a Multimodal Attention U-Net

Swaib Ilias Mazumder, Manish Kumar, Aparajita Khan

机构 * 1Computer Science \& Engineering, Indian Institute of Technology Roorkee, India 2Computer Science \& Engineering, Indian Institute of Technology Ropar, India 3 Computer Science \& Engineering, Indian Institute of Technology (BHU) Varanasi, India 纠错

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Comments Accepted in Geospatial AI and Applications with Foundation Models (GAIA) 2025, INSAIT and ELLIS Unit Sofia, Bulgaria

Better Safe Than Sorry? Overreaction Problem of Vision Language Models in Visual Emergency Recognition

Dasol Choi, Seunghyun Lee, Youngsook Song

机构 * AIM Intelligence（AIM智能公司） ； Yonsei University（延世大学） ； Lablup Inc.（Lablup公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQA

Yan Ke, Xin Yu, Heming Du, Scott Chapman, Helen Huang

机构 * The University of Queensland（昆士兰大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 13 pages, 2 figures, 2 tables

An Automated Retrieval-Augmented Generation LLaMA-4 109B-based System for Evaluating Radiotherapy Treatment Plans

Junjie Cui, Peilong Wang, Jason Holmes, Leshan Sun, Michael L. Hinni, Barbara A. Pockaj, Sujay A. Vora, Terence T. Sio, William W. Wong, Nathan Y. Yu, Steven E. Schild, Joshua R. Niska, Sameer R. Keole, Jean-Claude M. Rwigema, Samir H. Patel, Lisa A. McGee, Carlos A. Vargas, Wei Liu

机构 * Mayo Clinic Arizona（梅奥诊所亚利桑那分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 16 pages, 4 figures. Submitted to npj Digital Medicine

Semi-structured LLM Reasoners Can Be Rigorously Audited

Jixuan Leng, Cassandra A. Cohen, Zhixian Zhang, Chenyan Xiong, William W. Cohen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts

Michal Golovanevsky, William Rudman, Michael Lepori, Amir Bar, Ritambhara Singh, Carsten Eickhoff

机构 * Brown University（布朗大学） ； Tel Aviv University（特拉维夫大学） ； University of Tübingen（图宾根大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG