大模型推理能力

MTCMB: A Multi-Task Benchmark Framework for Evaluating LLMs on Knowledge, Reasoning, and Safety in Traditional Chinese Medicine

Shufeng Kong, Xingru Yang, Yuanyuan Wei, Zijie Wang, Hao Tang, Jiuqi Qin, Shuting Lan, Yingheng Wang, Junwen Bai, Zhuangbin Chen, Zibin Zheng, Caihua Liu, Hao Liang

机构 * School of Software Engineering, Sun Yat-sen University（中山大学软件学院） ； Institute of TCM Diagnostics, Hunan University of Chinese Medicine（湖南中医药大学中医诊断研究所） ； School of Artificial Intelligence, Guilin University of Electronic Technology（桂林电子科技大学人工智能学院） ； Department of Computer Science, Cornell University（康奈尔大学计算机科学系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.CL、cs.AI

RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios

Ruiwen Zhou, Wenyue Hua, Liangming Pan, Sitao Cheng, Xiaobao Wu, En Yu, William Yang Wang

机构 * University of California, Santa Barbara（加州大学圣芭芭拉分校） ； University of Arizona（亚利桑那大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Main Conference

From Reasoning to Generalization: Knowledge-Augmented LLMs for ARC Benchmark

Chao Lei, Nir Lipovetzky, Krista A. Ehinger, Yanchuan Chang

机构 * School of Computing and Information Systems, The University of Melbourne（计算与信息系统学院，墨尔本大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.CL、cs.AI

UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Models

Yu Zheng, Longyi Liu, Yuming Lin, Jie Feng, Guozhen Zhang, Depeng Jin, Yong Li

机构 * The Thørväld Group（Thørväld集团） ； Inria Paris-Rocquencourt（Inria巴黎-罗克桑court分部） ； Rajiv Gandhi University Doimukh（拉贾格恩迪大学多伊穆克） ； Tsinghua University（清华大学） ； Palmer Research Laboratories（帕勒默研究实验室） ； The Kumquat Consortium（Kumquat联盟） 纠错

专题命中 推理评测 ：planning(title,abstract)；reasoning(abstract)；分类 cs.CL、cs.AI

LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception

Yuan-Hong Liao, Sven Elflein, Liu He, Laura Leal-Taixé, Yejin Choi, Sanja Fidler, David Acuna

专题命中 推理评测 ：reasoning(title,abstract)；verifier(abstract)；分类 cs.CL、cs.LG

Comments 24 pages, 10 figures, in submission. Project page: https://andrewliao11.github.io/LongPerceptualThoughts

Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks

Rushang Karia, Daniel Bramblett, Daksh Dobhal, Siddharth Srivastava

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs

Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, Yihan Cao, Hui Ren, Xiang Li, Xiaoxiao Li, Yuyin Zhou

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 11 figures, 6 tables. Project page: https://github.com/UCSC-VLAA/MedReason

PORT: Preference Optimization on Reasoning Traces

Salem Lahlou, Abdalgader Abubaker, Hakim Hacid

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.LG

Inductive Linguistic Reasoning with Large Language Models

Raghav Ramji, Keshav Ramji

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs

Zhongshen Zeng, Yinhong Liu, Yingjia Wan, Jingyao Li, Pengguang Chen, Jianbo Dai, Yuxuan Yao, Rongwu Xu, Zehan Qi, Wanru Zhao, Linling Shen, Jianqiao Lu, Haochen Tan, Yukang Chen, Hao Zhang, Zhan Shi, Bailin Wang, Zhijiang Guo, Jiaya Jia

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Calibrating Reasoning in Language Models with Internal Consistency

Zhihui Xie, Jizhou Guo, Tong Yu, Shuai Li

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 camera ready

OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models

Junda Wu, Xintong Li, Ruoyu Wang, Yu Xia, Yuxin Xiong, Jianing Wang, Tong Yu, Xiang Chen, Branislav Kveton, Lina Yao, Jingbo Shang, Julian McAuley

专题命中 推理评测 ：chain-of-thought(title,abstract)；reasoning(abstract)；分类 cs.CL、cs.LG

Comments 10 pages

Step-by-Step Reasoning to Solve Grid Puzzles: Where do LLMs Falter?

Nemika Tyagi, Mihir Parmar, Mohith Kulkarni, Aswin RRV, Nisarg Patel, Mutsumi Nakamura, Arindam Mitra, Chitta Baral

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2024 Main

Can-Do! A Dataset and Neuro-Symbolic Grounded Framework for Embodied Planning with Large Multimodal Models

Yew Ken Chia, Qi Sun, Lidong Bing, Soujanya Poria

专题命中 推理评测 ：planning(title,abstract)；reasoning(abstract)；分类 cs.CL、cs.AI

LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models

Shibo Hao, Yi Gu, Haotian Luo, Tianyang Liu, Xiyan Shao, Xinyuan Wang, Shuhua Xie, Haodi Ma, Adithya Samavedhi, Qiyue Gao, Zhen Wang, Zhiting Hu

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments Project website: https://www.llm-reasoners.net/

VideoVista: A Versatile Benchmark for Video Understanding and Reasoning

Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, Min Zhang

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments 38 pages, 44 figures

Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data

Xiao Liu, Zirui Wu, Xueqing Wu, Pan Lu, Kai-Wei Chang, Yansong Feng

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments Findings of ACL 2024. Project website: https://xxxiaol.github.io/QRData/

STAR: A Benchmark for Situated Reasoning in Real-World Videos

Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, Chuang Gan

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS

Mixed Distillation Helps Smaller Language Model Better Reasoning

Chenglin Li, Qianglong Chen, Liangyue Li, Caiyu Wang, Yicheng Li, Zulong Chen, Yin Zhang

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL、cs.AI

Comments Working in Progress, 17 pages, 16 figures

Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark

Fangjun Li, David C. Hogg, Anthony G. Cohn

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments Camera-Ready version for AAAI 2024

CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, Sida I. Wang

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI、cs.LG

Comments 71 pages, 29 figures

A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL、cs.AI

Comments 45 pages, AACL 2023

SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables

Xinyuan Lu, Liangming Pan, Qian Liu, Preslav Nakov, Min-Yen Kan

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2023 (main conference, long paper)

Contrastive Decoding Improves Reasoning in Large Language Models

Sean O'Brien, Mike Lewis

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments 9 figures, 11 tables

Can GPT-3 Perform Statutory Reasoning?

Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments 10 pages

Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics

Mohan Sridharan, Tiago Mota

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.AI、cs.LG

Comments 37 pages, 17 figures, 5 tables

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

ROM：通过流式检测和干预实时抑制过度思考

Xinyan Wang, Xiaogeng Liu, Ming Pei, Chaowei Xiao

机构 * University of Wisconsin–Madison（威斯康星大学麦迪逊分校） ； Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract_cn)；self-correction(abstract)

AI总结 ROM通过流式检测和干预框架减少大推理模型的过度思考，提升准确率并缩短响应长度，在多个数据集上均取得显著效果。

Comments Code is available at https://github.com/SaFo-Lab/ROM

When Prompts Mislead: Textual Dominance and Diagnostic Bias in MLLMs

当提示误导：多模态大语言模型中的文本主导与诊断偏差

Inhyuk Park, Doohyun Park

专题命中 推理评测 ：CoT(abstract,abstract_cn)；reasoning(abstract)；chain-of-thought(abstract)；self-correction(abstract)

AI总结 研究揭示在医学多模态大语言模型中，文本提示会主导视觉线索，导致诊断偏差，即使模型具备空间定位能力，提示策略仍可能不安全。

Comments Accepted to the CVPR 2026 MMFM-BIOMED Workshop

When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

LLM何时推理？基于熵相变的动力系统视角

Wei Xia, Haoqing Wang, Zhi-Hong Deng, Yehui Tang

机构 * Samsung Research（三星研究院） ； State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（通用人工智能国家重点实验室，北京理工大学） 纠错

专题命中 推理评测 ：CoT(abstract,abstract_cn)；reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本文通过早期解码熵动态检测LLM的推理状态，提出轻量级无训练路由框架EDRM，自适应选择推理策略，在减少token消耗的同时提升准确率。

RoadTones: Tone Controllable Text Generation from Road Event Videos

RoadTones: 从道路事件视频生成可调节语气的文本

Chirag Parikh, Siddhi Pravin Lipare, Ravi Kiran Sarvadevabhatla

机构 * CVIT & iHub-Data, IIIT Hyderabad, India（CVIT与iHub-Data，IIIT海得拉巴，印度） 纠错

专题命中 推理评测 ：CoT(summary_cn,abstract)；chain-of-thought(abstract)

AI总结 本文提出RoadTones-51K数据集和RoadTones-VL-CoT模型，通过生成语气条件的推理草稿提升可解释性，并引入RoadTones-Eval评估体系，共同为上下文敏感的可调节视频描述奠定基础。

Comments Accepted at CVPR Findings 2026. Project page: https://roadtones.github.io/