大模型推理能力

Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning

Shulin Tian, Ruiqi Wang, Hongming Guo, Penghao Wu, Yuhao Dong, Xiuying Wang, Jingkang Yang, Hao Zhang, Hongyuan Zhu, Ziwei Liu

机构 * S-Lab, Nanyang Technological University（南洋理工大学S实验室） ； A*STAR, Singapore（新加坡A*STAR） ； Simon Fraser University（西蒙弗雷泽大学） ； Shanghai AI Lab（上海人工智能实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.AI

Comments Project page: https://egolife-ai.github.io/Ego-R1/

Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penalty

Zehui Ling, Deshu Chen, Hongwei Zhang, Yifeng Jiao, Xin Guo, Yuan Cheng

机构 * Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化院） ； School of Data Science, Fudan University（复旦大学数据科学学院） ； Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark

Ziming Cheng, Binrui Xu, Lisheng Gong, Zuhe Song, Tianshuo Zhou, Shiqi Zhong, Siyu Ren, Mingxiang Chen, Xiangchao Meng, Yuxin Zhang, Yanlin Li, Lei Ren, Wei Chen, Zhiyuan Huang, Mingjie Zhan, Xiaojie Wang, Fangxiang Feng

机构 * BUPT China（北京邮电大学） ； YSU China（云南大学） ； NUS Singapore（新加坡国立大学） ； Li Auto Inc. China（利汽车公司） ； SenseTime Research China（商汤研究） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

Comments 18 pages

More or Less Wrong: A Benchmark for Directional Bias in LLM Comparative Reasoning

Mohammadamin Shafiei, Hamidreza Saffari, Nafise Sadat Moosavi

机构 * University of Milan（米兰大学） ； Politecnico di Milano（米兰理工学院） ； University of Sheffield（谢菲尔德大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale

Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, Xiang Yue

机构 * Carnegie Mellon University（卡内基梅隆大学） ； M-A-P ； Nanyang Technological University（南洋理工大学） ； University of Waterloo（滑铁卢大学） ； The University of Manchester（曼彻斯特大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL

Comments ACL 2025 Main

Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models

Patrick Y. Wu, Jonathan Nagler, Joshua A. Tucker, Solomon Messing

机构 * American University（美国大学） ； New York University（纽约大学） ； Center for Social Media and Politics（社交媒体与政治中心） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；reasoning(abstract)；分类 cs.CL

Comments 10 pages, 2 figures. Appears in 2024 IEEE International Conference on Big Data (BigData). Please cite the published version: 10.1109/BigData62323.2024.10825235

Journal ref 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 7232-7241

Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?

Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, Mor Geva

机构 * Google DeepMind（谷歌DeepMind） ； UCL（伦敦大学学院） ； Google Research（谷歌研究） ； Tel Aviv University（特拉维夫大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Findings of ACL 2025

SocialMaze: A Benchmark for Evaluating Social Reasoning in Large Language Models

Zixiang Xu, Yanbo Wang, Yue Huang, Jiayi Ye, Haomin Zhuang, Zirui Song, Lang Gao, Chenxi Wang, Zhaorun Chen, Yujun Zhou, Sixian Li, Wang Pan, Yue Zhao, Jieyu Zhao, Xiangliang Zhang, Xiuying Chen

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Code available at https://github.com/xzx34/SocialMaze

SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis

Yansong Li, Paula Branco, Alexander M. Hoole, Manish Marwah, Hari Manassery Koduvely, Guy-Vincent Jourdan, Stephan Jou

机构 * University of Ottawa（渥太华大学） ； OpenText（OpenText公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

Journal ref 2025 IEEE Symposium on Security and Privacy (SP), 2025, pp. 2791-2809

ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant

Yifan Xiang, Zhenxi Zhang, Bin Li, Yixuan Weng, Shoujun Zhou, Yangfan He, Keqin Li

机构 * Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所） ； School of Engineering, Westlake University（西湖大学工程学院） ； University of Minnesota – Twin Cities（明尼苏达大学双城分校） ； University of Toronto（多伦多大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

Comments Work in progress

CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation

Ningxin Gui, Qianghuai Jia, Feijun Jiang, Yuling Jiao, dechun wang, Jerry Zhijian Yang

机构 * School of Mathematics and Statistics（数学与统计学学院） ； Alibaba International Digital Commerce（阿里巴巴国际数字商务） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

Rosetta-PL: Propositional Logic as a Benchmark for Large Language Model Reasoning

Shaun Baek, Shaun Esua-Mensah, Cyrus Tsui, Sejan Vigneswaralingam, Abdullah Alali, Michael Lu, Vasu Sharma, Sean O'Brien, Kevin Zhu

机构 * Emory University（埃默里大学） ； Algoverse AI Research（Algoverse AI研究） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge

Yueqi Song, Tianyue Ou, Yibo Kong, Zecheng Li, Graham Neubig, Xiang Yue

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

Comments 56 pages, 43 figures

Reasoning with LLMs for Zero-Shot Vulnerability Detection

Arastoo Zibaeirad, Marco Vieira

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.AI

Chain of Evidences and Evidence to Generate: Prompting for Context Grounded and Retrieval Augmented Reasoning

Md Rizwan Parvez

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL

Comments Accepted at NAACL KnowledgeNLP 2025

Guess What I am Thinking: A Benchmark for Inner Thought Reasoning of Role-Playing Language Agents

Rui Xu, MingYu Wang, XinTao Wang, Dakuan Lu, Xiaoyu Tan, Wei Chu, Yinghui Xu

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.AI

Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases

Pengcheng Qiu, Chaoyi Wu, Shuyu Liu, Weike Zhao, Zhuoxia Chen, Hongfei Gu, Chuanjin Peng, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.CL

MiCEval: Unveiling Multimodal Chain of Thought's Quality via Image Description and Reasoning Steps

Xiongtao Zhou, Jie He, Lanyu Chen, Jingyu Li, Haojing Chen, Víctor Gutiérrez-Basulto, Jeff Z. Pan, Hanjie Chen

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments NAACL 2025

Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning

Mohamed Aghzal, Erion Plaku, Ziyu Yao

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.CL

PEA: Enhancing LLM Performance on Computational-Reasoning Tasks

Zi Wang, Shiwei Weng, Mohannad Alhanahnah, Somesh Jha, Tom Reps

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.AI

MetRex: A Benchmark for Verilog Code Metric Reasoning Using LLMs

Manar Abdelatty, Jingxiao Ma, Sherief Reda

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.CL

Socratic Questioning: Learn to Self-guide Multimodal Reasoning in the Wild

Wanpeng Hu, Haodi Liu, Lin Chen, Feng Zhou, Changming Xiao, Qi Yang, Changshui Zhang

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

Do Multimodal Language Models Really Understand Direction? A Benchmark for Compass Direction Reasoning

Hang Yin, Zhifeng Lin, Xin Liu, Bin Sun, Kan Li

专题命中 推理评测 ：reasoning(title,abstract)；CoT(abstract)；分类 cs.AI

Reasoning-Aware Query-Focused Summarization over Multi-Table Data

Xiaochuan Lin, Xiangyong Chen

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge

Haotong Zhang

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations

Jiaxing Sun, Weiquan Huang, Jiang Wu, Chenya Gu, Wei Li, Songyang Zhang, Hang Yan, Conghui He

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Equal contribution: Jiaxing Sun, Weiquan Huang, Jiang Wu; Corresponding author: Conghui He

The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights

Wenhao Zhu, Shujian Huang, Fei Yuan, Cheng Chen, Jiajun Chen, Alexandra Birch

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs

Houman Mehrafarin, Arash Eshghi, Ioannis Konstas

专题命中 推理评测 ：reasoning(title,abstract)；logical reasoning(abstract)；分类 cs.CL

Comments To appear in EMNLP Main 2024

MalAlgoQA: Pedagogical Evaluation of Counterfactual Reasoning in Large Language Models and Implications for AI in Education

Naiming Liu, Shashank Sonkar, Myco Le, Richard Baraniuk

专题命中 推理评测 ：reasoning(title,abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2024 Findings Track

Towards a Benchmark for Causal Business Process Reasoning with LLMs

Fabiana Fournier, Lior Limonad, Inna Skarbovsky

专题命中 推理评测 ：reasoning(title,abstract)；planning(abstract)；分类 cs.AI

Comments 12 pages, 1 figure

Journal ref NLP4BPM workshop at BPM 2024