大模型推理能力

MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents

Xuehui Wang, Zhenyu Wu, JingJing Xie, Zichen Ding, Bowen Yang, Zehao Li, Zhaoyang Liu, Qingyun Li, Xuan Dong, Zhe Chen, Weiyun Wang, Xiangyu Zhao, Jixuan Chen, Haodong Duan, Tianbao Xie, Chenyu Yang, Shiqian Su, Yue Yu, Yuan Huang, Yiqian Liu, Xiao Zhang, Yanting Zhang, Xiangyu Yue, Weijie Su, Xizhou Zhu, Wei Shen, Jifeng Dai, Wenhai Wang

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Shanghai Jiao Tong University（上海交通大学） ； Xiamen University（厦门大学） ； University of Science and Technology of China（中国科学技术大学） ； The Hong Kong University of Science and Technology（香港科技大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； Tsinghua University（清华大学） ； Nanjing University（南京大学） ； Fudan University（复旦大学） ； University of Hong Kong（香港大学） ； Donghua University（东华大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

Comments in progress

Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering

William Jurayj, Jeffrey Cheng, Benjamin Van Durme

机构 * Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；test-time compute(abstract)；分类 cs.CL

Comments Accepted to ACL 2025. Code: https://github.com/wjurayj/final_answer

ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviews

Xian Gao, Jiacheng Ruan, Zongyun Zhang, Jingsheng Gao, Ting Liu, Yuzhuo Fu

机构 * Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments Work in progress

GraphRunner: A Multi-Stage Framework for Efficient and Accurate Graph-Based Retrieval

Savini Kashmira, Jayanaka L. Dantanarayana, Krisztián Flautner, Lingjia Tang, Jason Mars

机构 * University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale

Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, Hong Chen, Cuiping Li

机构 * Renmin University of China（中国人民大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Rethinking Verification for LLM Code Generation: From Generation to Testing

Zihan Ma, Taolin Zhang, Maosong Cao, Junnan Liu, Wenwei Zhang, Minnan Luo, Songyang Zhang, Kai Chen

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院） ； MOE KLINNS Lab, Xi’an Jiaotong University（西安交通大学教育部KLINNS实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；verifier(abstract)；分类 cs.CL

DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthiness

Ahmad Mohammadshirazi, Pinaki Prasad Guha Neogi, Ser-Nam Lim, Rajiv Ramnath

机构 * Department of Computer Science（计算机科学系） ； Engineering, Ohio State University, Ohio, US（工程系，俄亥俄州立大学，俄亥俄，美国） ； Department of Computer Science, University of Central Florida, Florida, US（计算机科学系，中央佛罗里达大学，佛罗里达，美国） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation

Guangya Yu, Yanhao Li, Zongying Jiang, Yuxiong Jin, Li Dai, Yupian Lin, Ruihui Hou, Weiyan Zhang, Yongqi Fan, Qi Ye, Jingping Liu, Tong Ruan

机构 * School of Information Science and Engineering, East China University of Science and Technology（信息科学与工程学院，东华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 2025 ACL Findings

NoLiMa: Long-Context Evaluation Beyond Literal Matching

Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Trung Bui, Ryan A. Rossi, Seunghyun Yoon, Hinrich Schütze

机构 * LMU（莱茵-穆尔大学） ； Center for Information and Language Processing（信息与语言处理中心） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心） ； Adobe（Adobe公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments Accepted at ICML 2025

CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs

Haoxi Li, Sikai Bai, Jie Zhang, Song Guo

机构 * HKUST（香港科技大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.LG

Comments 9 pages, 6 figures

LLMs are Introvert

Litian Zhang, Xiaoming Zhang, Bingyu Yan, Ziyi Zhou, Bo Zhang, Zhenyu Guan, Xi Zhang, Chaozhuo Li

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Beihang University（北京航空航天大学） ； Nanjing Normal University（南京师范大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.AI

An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability

Yusuke Yamauchi, Taro Yano, Masafumi Oyamada

机构 * The University of Tokyo（东京大学） ； NEC Corporation（日本电讯公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Prompt engineering and framework: implementation to increase code reliability based guideline for LLMs

Rogelio Cruz, Jonatan Contreras, Francisco Guerrero, Ezequiel Rodriguez, Carlos Valdez, Citlali Carrillo

机构 * IBM Technologies（IBM技术） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs

Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, Linjie Li, Furong Huang, Lijuan Wang

机构 * University of Maryland College Park（马里兰大学学院市分校） ； Microsoft（微软） ； University of Michigan（密歇根大学） ； Cardiff University（卡迪夫大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.LG

Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training

Shurui Gui, Shuiwang Ji

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.LG

Hidden in Plain Sight: Evaluation of the Deception Detection Capabilities of LLMs in Multimodal Settings

Md Messal Monem Miah, Adrita Anika, Xi Shi, Ruihong Huang

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted to ACL 2025 Main Conference

Detecting Voice Phishing with Precision: Fine-Tuning Small Language Models

Ju Yong Sim, Seong Hwan Kim

机构 * Korea National University of Transportation（韩国交通大学） 纠错

专题命中 推理评测 ：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments 15 pages, 4 figures, 8 tables, journal submission

MiMo-VL Technical Report

Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai, Bingquan Xia

机构 * LLM-Core ； Xiaomi（小米） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 32 pages

Climate Finance Bench

Rafik Mankour, Yassine Chafai, Hamada Saleh, Ghassen Ben Hassine, Thibaud Barreau, Peter Tankov

机构 * Institut Louis Bachelier（路易·巴舍利尔研究所） ； CREST, ENSAE, Institut Polytechnique de Paris（CREST、ENSAE、巴黎理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Dataset is available at https://github.com/Pladifes/climate_finance_bench

From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications

Feibo Jiang, Cunhua Pan, Li Dong, Kezhi Wang, Octavia A. Dobre, Merouane Debbah

机构 * Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, Hunan Normal University（智能计算与语言信息处理湖南省重点实验室，湖南师范大学） ； National Mobile Communications Research Laboratory, Southeast University（国家移动通信研究实验室，东南大学） ； Changsha Social Laboratory of Artificial Intelligence, Hunan University of Technology and Business（长沙人工智能社会实验室，湖南工业大学） ； Department of Computer Science, Brunel University London（伦敦布鲁内尔大学计算机科学系） ； Faculty of Engineering and Applied Science, Memorial University（纪念大学工程与应用科学学院） ； G Research Center, Khalifa University of Science and Technology（科技大学6G研究中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

Towards Reliable Large Audio Language Model

Ziyang Ma, Xiquan Li, Yakun Song, Wenxi Chen, Chenpeng Du, Jian Wu, Yuanzhe Chen, Zhuo Chen, Yuping Wang, Yuxuan Wang, Xie Chen

机构 * X-LANCE Lab, School of Computer Science, MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University（X-LANCE实验室，计算机科学学院，人工智能国家重点实验室，上海交通大学） ； ByteDance（字节跳动） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments ACL 2025 Findings

CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space

Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, Jincai Huang

机构 * National University of Defense Technology（国防科技大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments ACL 2025 Main

RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward

Zongsheng Wang, Kaili Sun, Bowen Wu, Qun Yu, Ying Li, Baoxun Wang

机构 * Platform and Content Group, Tencent（腾讯平台与内容部） ； School of Software & Microelectronics, Peking University（北京大学软件与微电子学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Towards Autonomous UAV Visual Object Search in City Space: Benchmark and Agentic Methodology

Yatai Ji, Zhengqiu Zhu, Yong Zhao, Beidan Liu, Chen Gao, Yihao Zhao, Sihang Qiu, Yue Hu, Quanjun Yin, Yong Li

机构 * State Key Lab of Digital-Intelligent Modeling and Simulation（数字智能建模与仿真国家重点实验室） ； Department of Electronic Engineering, Tsinghua University（清华大学电子工程系） 纠错

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.AI

POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications

Chunjing Gan, Dan Yang, Binbin Hu, Ziqi Liu, Yue Shen, Zhiqiang Zhang, Jian Wang, Jun Zhou

机构 * Ant Group（蚂蚁集团） 纠错

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.LG

OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs

John Murzaku, Owen Rambow

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Submitted to COLM 2025. Preprint

Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving

Yue Li, Meng Tian, Zhenyu Lin, Jiangtong Zhu, Dechang Zhu, Haiqiang Liu, Zining Wang, Yueyi Zhang, Zhiwei Xiong, Xinhai Zhao

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.CL

LongViTU: Instruction Tuning for Long-Form Video Understanding

Rujie Wu, Xiaojian Ma, Hai Ci, Yue Fan, Yuxuan Wang, Haozhe Zhao, Qing Li, Yizhou Wang

专题命中 推理评测 ：reasoning(abstract)；planning(abstract)；分类 cs.LG

(G)I-DLE: Generative Inference via Distribution-preserving Logit Exclusion with KL Divergence Minimization for Constrained Decoding

Hanwool Lee

专题命中 推理评测 ：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments preprint