大模型推理能力

Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluation

Jungkoo Kang

机构 * IBM Research（IBM研究院） ； MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室） 纠错

专题命中 推理评测 ：planning(title,abstract)；reasoning(abstract)；分类 cs.AI

Comments 30 pages, 7 figures

The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models

Siqi Fan, Bowen Qin, Peng Han, Shuo Shang, Yequan Wang, Aixin Sun

机构 * University of Electronic Science and Technology of China（电子科技大学） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL

Comments Added new experiments and revised the manuscript for clarity

LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models

Yiwei Liu, Yucheng Li, Xiao Li, Gong Cheng

机构 * State Key Laboratory for Novel Software Technology（新型软件技术国家重点实验室） ； Nanjing University（南京大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

Comments 30 pages, 3 figures

Unleashing Perception-Time Scaling to Multimodal Reasoning Models

Yifan Li, Zhenghao Chen, Ziheng Wu, Kun Zhou, Ruipu Luo, Can Zhang, Zhentao He, Yufei Zhan, Wayne Xin Zhao, Minghui Qiu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院） ； Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理重点实验室） ； ByteDance（字节跳动） ； University of California, San Diego（加州大学圣地亚哥分校） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；math reasoning(abstract)；分类 cs.CL

SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models

Andong Deng, Taojiannan Yang, Shoubin Yu, Lincoln Spencer, Mohit Bansal, Chen Chen, Serena Yeung-Levy, Xiaohan Wang

机构 * University of Central Florida（中央佛罗里达大学） ； University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校） ； Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.AI

MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning

Neeraja Kirtane, Yuvraj Khanna, Peter Relan

机构 * Got It Education（Got It 教育） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；math reasoning(abstract)；分类 cs.CL

Reasoning Models are Test Exploiters: Rethinking Multiple-Choice

Narun Raman, Taylor Lundy, Kevin Leyton-Brown

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 9 pages, 4 figures

OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learning

Xiao Zhang, Huiyuan Lai, Qianru Meng, Johan Bos

机构 * University of Groningen（格罗宁根大学） ； Leiden University（莱顿大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Grounding

Haotian Xue, Yunhao Ge, Yu Zeng, Zhaoshuo Li, Ming-Yu Liu, Yongxin Chen, Jiaojiao Fan

机构 * Georgia Tech（佐治亚理工学院） ； NVIDIA（英伟达） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.AI

ARK-V1: An LLM-Agent for Knowledge Graph Question Answering Requiring Commonsense Reasoning

Jan-Felix Klein, Lars Ohnemus

机构 * Institute of Material Handling and Logistics (IFL) at the Karlsruhe Institute of Technology (KIT)（材料搬运与物流研究所（IFL）于卡尔斯鲁厄理工学院（KIT）） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Work in Progess

D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models

Satyapriya Krishna, Andy Zou, Rahul Gupta, Eliot Krzysztof Jones, Nick Winter, Dan Hendrycks, J. Zico Kolter, Matt Fredrikson, Spyros Matsoukas

机构 * Amazon Nova Responsible AI（亚马逊Nova负责任的人工智能） ； Center for AI Safety（人工智能安全中心） ； CMU（卡内基梅隆大学） ； Gray Swan AI（灰天鹅人工智能） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Preprint

TurnaboutLLM: A Deductive Reasoning Benchmark from Detective Games

Yuan Yuan, Muyu He, Muhammad Adil Shahid, Jiani Huang, Ziyang Li, Li Zhang

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments In EMNLP 2025 main conference

Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Reasoning

Mingyuan Wu, Jize Jiang, Haozhen Zheng, Meitang Li, Zhaoheng Li, Beitong Tian, Bo Chen, Yongjoo Park, Minjia Zhang, Chengxiang Zhai, Klara Nahrstedt

机构 * University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校） ； University of Michigan Ann Arbor（密歇根大学安娜堡分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.LG

Comments EMNLP 2025 Main Conference. Mingyuan, Jize, and Haozhen contributed equally, while Minjia, Chengxiang, and Klara advised equally

Reasoning with Preference Constraints: A Benchmark for Language Models in Many-to-One Matching Markets

Marylou Fauchard, Florian Carichon, Margarida Carvalho, Golnoosh Farnadi

机构 * Université de Montréal（蒙特利尔大学） ； Mila ； McGill（麦吉尔大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.AI

FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark

Rongyao Fang, Aldrich Yu, Chengqi Duan, Linjiang Huang, Shuai Bai, Yuxuan Cai, Kun Wang, Si Liu, Xihui Liu, Hongsheng Li

机构 * CUHK（香港中文大学） ； HKU（香港大学） ； BUAA（北京航空航天大学） ； Alibaba（阿里巴巴） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Project page: https://flux-reason-6m.github.io/

StoryReasoning Dataset: Using Chain-of-Thought for Scene Understanding and Grounded Story Generation

Daniel A. P. Oliveira, David Martins de Matos

机构 * Instituto Superior Técnico, Universidade de Lisboa（里斯本大学技术学院） ； INESC-ID Lisboa（里斯本INESC-ID） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；reasoning(abstract)；分类 cs.CL

Comments 31 pages, 14 figures

Investigating Advanced Reasoning of Large Language Models via Black-Box Interaction

Congchi Yin, Tianyi Wu, Yankai Shu, Alex Gu, Yunhan Wang, Jun Shao, Xun Jiang, Piji Li

机构 * College of Artificial Intelligence, Nanjing University of Aeronautics（人工智能学院，南京航空航天大学） ； School of Electronics Engineering（电子工程学院） ； Computer Science, Peking University（计算机科学，北京大学） ； Theta Health Inc.（Theta健康公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.AI

DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation

Abdelrahman Abdallah, Jamshid Mozafari, Bhawna Piryani, Adam Jatowt

机构 * University of Innsbruck（因斯布鲁克大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accept at EMNLP Findings 2025

HoarePrompt: Structural Reasoning About Program Correctness in Natural Language

Dimitrios Stamatios Bouras, Yihan Dai, Tairan Wang, Yingfei Xiong, Sergey Mechtaev

机构 * Peking University（北京大学） ； Nankai University（南开大学） ； University College London（伦敦大学学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

A Multi-Agent Approach to Neurological Clinical Reasoning

Moran Sorka, Alon Gorenshtein, Dvir Aran, Shahar Shelly

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.AI

Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation

Yuheng Zha, Kun Zhou, Yujia Wu, Yushu Wang, Jie Feng, Zhi Xu, Shibo Hao, Zhengzhong Liu, Eric P. Xing, Zhiting Hu

机构 * UC San Diego（UC圣地亚哥大学） ； Carnegie Mellon University（卡内基梅隆大学） ； MBZUAI（马斯克人工智能研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection

Yanhui Li, Yunkang Cao, Chengliang Liu, Yuan Xiong, Xinghui Dong, Chao Huang

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.AI

GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environments

Kechen Li, Yaotian Tao, Ximing Wen, Quanwei Sun, Zifei Gong, Chang Xu, Xizhe Zhang, Tianbo Ji

机构 * School of Transportation and Civil Engineering, Nantong University, China（南京理工大学交通与土木工程学院） ； HongKong Linchance Intelligent Technology Co., Ltd. (LinChance), China（香港林 chance 智能科技有限公司） ； College of Computing and Informatics, Drexel University, USA（德雷塞尔大学计算机与信息学院） ； University of New South Wales, Australia（新南威尔士大学） 纠错

专题命中 推理评测 ：planning(title,abstract)；reasoning(abstract)；分类 cs.AI

Comments 8 pages

TripTailor: A Real-World Benchmark for Personalized Travel Planning

Yuanzhe Shen, Kaimin Wang, Changze Lv, Xiaoqing Zheng, Xuanjing Huang

机构 * School of Computer Science, Fudan University（复旦大学计算机科学学院） 纠错

专题命中 推理评测 ：planning(title,abstract)；reasoning(abstract)；分类 cs.AI

Comments Accepted to ACL 2025 Findings

ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans

Ananya Sadana, Yash Kumar Lal, Jiawei Zhou

机构 * Stony Brook University（石溪大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains

Xuzhao Li, Xuchen Li, Shiyu Hu, Yongzhen Guo, Wentao Zhang

专题命中 推理评测 ：reasoning(title,abstract)；verifier(abstract)；分类 cs.AI

Comments Preprint, Under review

TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning

Xinyuan Lu, Liangming Pan, Yubo Ma, Preslav Nakov, Min-Yen Kan

机构 * National University of Singapore（新加坡国立大学） ； University of Arizona（亚利桑那大学） ； Nanyang Technology University（南洋理工大学） ； MBZUAI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments NAACL 2025 (Findings)

Effects of structure on reasoning in instance-level Self-Discover

Sachith Gunasekara, Yasiru Ratnayake

机构 * Surge Global（Surge全球） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

ReasonBridge: Efficient Reasoning Transfer from Closed to Open-Source Language Models

Ziqi Zhong, Xunzhu Tang

机构 * London School of Economics（伦敦经济学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；test-time compute(abstract)；分类 cs.AI

DAVID-XR1: Detecting AI-Generated Videos with Explainable Reasoning

Yifeng Gao, Yifan Ding, Hongyu Su, Juncheng Li, Yunhan Zhao, Lin Luo, Zixing Chen, Li Wang, Xin Wang, Yixu Wang, Xingjun Ma, Yu-Gang Jiang

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.AI