大模型推理能力

MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation

Chenghao Yang, Yinbo Luo, Zhoufutu Wen, Qi Chu, Tao Gong, Longxiang Liu, Kaiyuan Zhang, Jianpeng Jiao, Ge Zhang, Wenhao Huang, Nenghai Yu

机构 * University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 29 pages, 13 figures, Accepted as EMNLP2025 Findings

Smart Trial: Evaluating the Use of Large Language Models for Recruiting Clinical Trial Participants via Social Media

Xiaofan Zhou, Zisu Wang, Janice Krieger, Mohan Zalake, Lu Cheng

机构 * University of Illinois at Chicago（伊利诺伊大学香槟分校） ； Colorado State University（科罗拉多州立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Temporal Preferences in Language Models for Long-Horizon Assistance

Ali Mazyaki, Mohammad Naghizadeh, Samaneh Ranjkhah Zonouzaghi, Hossein Setareh

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization

Zhengzhao Lai, Youbin Zheng, Zhenyang Cai, Haonan Lyu, Jinpu Yang, Hongqing Liang, Yan Hu, Benyou Wang

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Scaling Truth: The Confidence Paradox in AI Fact-Checking

Ihsan A. Qazi, Zohaib Khan, Abdullah Ghani, Agha A. Raza, Zafar A. Qazi, Wassay Sajjad, Ayesha Ali, Asher Javaid, Muhammad Abdullah Sohail, Abdul H. Azeemi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 65 pages, 26 figures, 6 tables

CTourLLM: Enhancing LLMs with Chinese Tourism Knowledge

Qikai Wei, Mingzhi Yang, Jinqiang Wang, Wenwei Mao, Jiabo Xu, Huansheng Ning

专题命中 推理评测 ：planning(abstract)；分类 cs.CL、cs.AI

RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use

Jiajun Chai, Guojun Yin, Zekun Xu, Chuhuai Yue, Yi Jia, Siyu Xia, Xiaohan Wang, Jiwen Jiang, Xiaoguang Li, Chengqi Dong, Hang He, Wei Lin

机构 * RLFactory Team（RLFactory团队） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

A Knowledge-Guided Cross-Modal Feature Fusion Model for Local Traffic Demand Prediction

Lingyu Zhang, Pengfei Xu, Guobin Wu, Jian Liang, Ruiyang Dong, Yunhai Wang, Xuan Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

BeSimulator: A Large Language Model Powered Text-based Behavior Simulator

Jianan Wang, Bin Li, Jingtao Qi, Xueying Wang, Fu Li, Hanxun Li

机构 * College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学） ； Intelligent Game and Decision Lab (IGDL)（智能游戏与决策实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 5 figures, 8 tables

ResearchArena: Benchmarking Large Language Models' Ability to Collect and Organize Information as Research Agents

Hao Kang, Chenyan Xiong

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Transforming Wearable Data into Personal Health Insights using Large Language Model Agents

Mike A. Merrill, Akshay Paruchuri, Naghmeh Rezaei, Geza Kovacs, Javier Perez, Yun Liu, Erik Schenck, Nova Hammerquist, Jake Sunshine, Shyam Tailor, Kumar Ayush, Hao-Wei Su, Qian He, Cory Y. McLean, Mark Malhotra, Shwetak Patel, Jiening Zhan, Tim Althoff, Daniel McDuff, Xin Liu

机构 * Google Research（谷歌研究） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 53 pages, 7 main figures, 2 main tables, accepted to Nature Communications

UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning

Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Liang, Shijue Huang, Wanjun Zhong, Yining Ye, Yujia Qin, Yuwen Xiong, Yuxin Song, Zhiyong Wu, Aoyan Li, Bo Li, Chen Dun, Chong Liu, Daoguang Zan, Fuxing Leng, Hanbin Wang, Hao Yu, Haobin Chen, Hongyi Guo, Jing Su, Jingjia Huang, Kai Shen, Kaiyu Shi, Lin Yan, Peiyao Zhao, Pengfei Liu, Qinghao Ye, Renjie Zheng, Shulin Xin, Wayne Xin Zhao, Wen Heng, Wenhao Huang, Wenqian Wang, Xiaobo Qin, Yi Lin, Youbin Wu, Zehui Chen, Zihao Wang, Baoquan Zhong, Xinchun Zhang, Xujing Li, Yuanfan Li, Zhongkai Zhao, Chengquan Jiang, Faming Wu, Haotian Zhou, Jinlin Pang, Li Han, Qi Liu, Qianli Ma, Siyao Liu, Songhua Cai, Wenqi Fu, Xin Liu, Yaohui Wang, Zhi Zhang, Bo Zhou, Guoliang Li, Jiajun Shi, Jiale Yang, Jie Tang, Li Li, Qihua Han, Taoran Lu, Woyu Lin, Xiaokang Tong, Xinyao Li, Yichi Zhang, Yu Miao, Zhengxuan Jiang, Zili Li, Ziyuan Zhao, Chenxin Li, Dehua Ma, Feng Lin, Ge Zhang, Haihua Yang, Hangyu Guo, Hongda Zhu, Jiaheng Liu, Junda Du, Kai Cai, Kuanye Li, Lichen Yuan, Meilan Han, Minchao Wang, Shuyue Guo, Tianhao Cheng, Xiaobo Ma, Xiaojun Xiao, Xiaolong Huang, Xinjie Chen, Yidi Du, Yilin Chen, Yiwen Wang, Zhaojian Li, Zhenzhu Yang, Zhiyuan Zeng, Chaolin Jin, Chen Li, Hao Chen, Haoli Chen, Jian Chen, Qinghao Zhao, Guang Shi

机构 * ByteDance Seed（字节跳动种子） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction

Zhiyuan Zeng, Jiashuo Liu, Siyuan Chen, Tianci He, Yali Liao, Yixiao Tian, Jinpeng Wang, Zaiyuan Wang, Yang Yang, Lingyue Yin, Mingren Yin, Zhenwei Zhu, Tianle Cai, Zehui Chen, Jiecao Chen, Yantao Du, Xiang Gao, Jiacheng Guo, Liang Hu, Jianpeng Jiao, Xiangsheng Li, Jingkai Liu, Shuang Ni, Zhoufutu Wen, Ge Zhang, Kaiyuan Zhang, Xin Zhou, Jose Blanchet, Xipeng Qiu, Mengdi Wang, Wenhao Huang

机构 * Fudan University（复旦大学） ； Stanford University（斯坦福大学） ； Princeton University（普林斯顿大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Technical report, 51 pages. Update the results

RAVEN: Query-Guided Representation Alignment for Question Answering over Audio, Video, Embedded Sensors, and Natural Language

Subrata Biswas, Mohammad Nur Hossain Khan, Bashima Islam

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Behavioral Fingerprinting of Large Language Models

Zehua Pei, Hui-Ling Zhen, Ying Zhang, Zhiyuan Yang, Xing Li, Xianzhi Yu, Mingxuan Yuan, Bei Yu

机构 * The Chinese University of Hong Kong（香港中文大学） ； Noah’s Ark Lab, Huawei（华为诺亚实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Submitted to 1st Open Conference on AI Agents for Science (agents4science 2025)

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling

Shengyin Sun, Yiming Li, Xing Li, Yingzhao Lian, Weizhe Lin, Hui-Ling Zhen, Zhiyuan Yang, Chen Chen, Xianzhi Yu, Mingxuan Yuan, Chen Ma

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 18 pages

Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models

Juraj Vladika, Mahdi Dhaini, Florian Matthes

机构 * Technical University of Munich（慕尼黑技术大学） ； School of Computation, Information and Technology（计算、信息与技术学院） ； Department of Computer Science（计算机科学系） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to Findings of EMNLP 2025

Measuring How (Not Just Whether) VLMs Build Common Ground

Saki Imai, Mert İnan, Anthony Sicilia, Malihe Alikhani

机构 * Northeastern University（东北大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Modular Techniques for Synthetic Long-Context Data Generation in Language Model Training and Evaluation

Seganrasan Subramanian, Abhigya Verma

机构 * ServiceNow 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 4 figures

MiniCPM4: Ultra-Efficient LLMs on End Devices

MiniCPM Team, Chaojun Xiao, Yuxuan Li, Xu Han, Yuzhuo Bai, Jie Cai, Haotian Chen, Wentong Chen, Xin Cong, Ganqu Cui, Ning Ding, Shengda Fan, Yewei Fang, Zixuan Fu, Wenyu Guan, Yitong Guan, Junshao Guo, Yufeng Han, Bingxiang He, Yuxiang Huang, Baoxi Ji, Cunliang Kong, Qiuzuo Li, Siyuan Li, Wenhao Li, Xin Li, Yanghao Li, Yishan Li, Zhen Li, Dan Liu, Biyuan Lin, Yankai Lin, Xiang Long, Quanyu Lu, Yaxi Lu, Peiyan Luo, Hongya Lyu, Litu Ou, Yinxu Pan, Lushi Pu, Zekai Qu, Qundong Shi, Zijun Song, Jiayuan Su, Zhou Su, Ao Sun, Xianghui Sun, Peijun Tang, Fangzheng Wang, Feng Wang, Shuo Wang, Yudong Wang, Zheng Wang, Yesai Wu, Zhenyu Xiao, Jie Xie, Zihao Xie, Xiaoyue Xu, Yukun Yan, Jiarui Yuan, Jinqian Zhang, Kaihuo Zhang, Lei Zhang, Linyue Zhang, Xueren Zhang, Yudi Zhang, Hengyu Zhao, Weilin Zhao, Weilun Zhao, Yuanqian Zhao, Zhi Zheng, Chuyue Zhou, Ge Zhou, Jie Zhou, Wei Zhou, Yanghao Zhou, Zihan Zhou, Zixuan Zhou, Zhiyuan Liu, Guoyang Zeng, Chao Jia, Dahai Li, Maosong Sun

机构 * MiniCPM Team（MiniCPM团队） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments MiniCPM4 Technical Report

Short-video Propagation Influence Rating: A New Real-world Dataset and A New Large Graph Model

Dizhan Xue, Shengsheng Qian, Chuanrui Hu, Changsheng Xu

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学） ； Qihoo 360 AI lab（奇虎360 AI实验室） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

AHELM: A Holistic Evaluation of Audio-Language Models

Tony Lee, Haoqin Tu, Chi Heem Wong, Zijun Wang, Siwei Yang, Yifan Mai, Yuyin Zhou, Cihang Xie, Percy Liang

机构 * Stanford University（斯坦福大学） ； University of California, Santa Cruz（加州大学圣克ruz分校） ； Hitachi America, Ltd.（日立美国有限公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward

Xinyu Tang, Zhenduo Zhang, Yurou Liu, Wayne Xin Zhao, Zujie Wen, Zhiqiang Zhang, Jun Zhou

机构 * Gaoling School of Artificial Intelligence, Renmin University of China 2 Ant Group（中国人民大学北京校区人工智能学院，中国人民大学 2 抗集团） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning

Yu Liu, Yanan Cao, Xixun Lin, Yanmin Shang, Shi Wang, Shirui Pan

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） ； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院） ； Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） ； Griffith University（格里菲斯大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025, Main, Long Paper

A Multi-granularity Concept Sparse Activation and Hierarchical Knowledge Graph Fusion Framework for Rare Disease Diagnosis

Mingda Zhang, Na Zhao, Jianglong Qin, Guoyu Ye, Ruixiang Tang

机构 * School of Software, Yunnan University（云南大学软件学院） ； Yunnan Key Laboratory of Software Engineering（云南软件工程重点实验室） ； Yunnan Provincial Hospital of Traditional Chinese Medicine（云南中医药省人民医院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 12 pages,3 figures

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation

Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

机构 * College of Computer Science and Artificial Intelligence, Fudan University（计算机科学与人工智能学院，复旦大学） ； Shanghai Key Lab of Intelligent Information Processing（上海智能信息处理重点实验室） ； Shanghai AI Laboratory（上海人工智能实验室） ； Northwestern University（西北大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanism

Jia Liu, ChangYi He, YingQiao Lin, MingMin Yang, FeiYang Shen, ShaoGuo Liu

机构 * Beihang University（北京航空航天大学） ； Northwestern Polytechnical University（西北工业大学） ； Kuaishou Technology（快手科技） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

A Graph Talks, But Who's Listening? Rethinking Evaluations for Graph-Language Models

Soham Petkar, Hari Aakash K, Anirudh Vempati, Akshit Sinha, Ponnurangam Kumarauguru, Chirag Agarwal

机构 * plaksha.edu.in（普拉克斯哈大学） ； research.iiit.ac.in（IIIT研究机构） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks

Aritra Dutta, Swapnanil Mukherjee, Deepanway Ghosal, Somak Aditya

机构 * IIT Kharagpur（印度理工学院Kharagpur分校） ； Ashoka University（阿什oka大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

SoAy: A Solution-based LLM API-using Methodology for Academic Information Seeking

Yuanchun Wang, Jifan Yu, Zijun Yao, Jing Zhang, Yuyang Xie, Shangqing Tu, Yiyang Fu, Youhe Feng, Jinkai Zhang, Jingyao Zhang, Bowen Huang, Yuanyao Li, Huihui Yuan, Lei Hou, Juanzi Li, Jie Tang

机构 * Renmin University of China（中国人民大学） ； Key Laboratory of Data Engineering and Knowledge Engineering, MOE（数据工程与知识工程重点实验室） ； Tsinghua University（清华大学） ； Institute of Education（教育学院） ； Department of Computer Science and Technology（计算机科学与技术系） ； Engineering Research Center of Database and Business Intelligence, MOE（数据库与商务智能工程研究中心） ； School of Information（信息学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments KDD 2025; 22 pages, 13 figures