大模型推理能力

CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward

Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F. Wong, Songyang Zhang, Kai Chen

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； NLP 2 CT Lab, University of Macau（澳门大学自然语言处理与计算机视觉实验室） ； University of Macau（澳门大学） 纠错

专题命中 推理评测 ：verifier(title,abstract)；reasoning(abstract)；分类 cs.CL、cs.AI

Comments Technical Report; 31 Pages

REALM-Bench: A Benchmark for Evaluating Multi-Agent Systems on Real-world, Dynamic Planning and Scheduling Tasks

Longling Geng, Edward Y. Chang

机构 * Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.AI

Comments 24 pages, 8 figures, 28 tables, 7 listings

Coherent Multimodal Reasoning with Iterative Self-Evaluation for Vision-Language Models

Wenjie Luo, Ruocheng Li, Shanshan Zhu, Julian Perry

机构 * Fujian University of Technology（福建工程大学） ； Delta University for Science and Technology（科技大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

The Multi-Round Diagnostic RAG Framework for Emulating Clinical Reasoning

Penglei Sun, Yixiang Chen, Xiang Li, Xiaowen Chu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

VideoForest: Person-Anchored Hierarchical Reasoning for Cross-Video Question Answering

Yiran Meng, Junhong Ye, Wei Zhou, Guanghui Yue, Xudong Mao, Ruomei Wang, Baoquan Zhao

机构 * Sun Yat-Sen University（中山大学） ； Cardiff University（卡地夫大学） ； Shenzhen University（深圳大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)

LaTCoder: Converting Webpage Design to Code with Layout-as-Thought

Yi Gui, Zhen Li, Zhongyi Zhang, Guohao Wang, Tianpeng Lv, Gaoyang Jiang, Yi Liu, Dongping Chen, Yao Wan, Hongyu Zhang, Wenbin Jiang, Xuanhua Shi, Hai Jin

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)

Comments KDD 2025 v2

Hide and Seek with LLMs: An Adversarial Game for Sneaky Error Generation and Self-Improving Diagnosis

Rui Zou, Mengqi Wei, Yutao Zhu, Jirong Wen, Xin Zhao, Jing Chen

专题命中 推理评测 ：reasoning(abstract)；math reasoning(abstract)；分类 cs.AI

Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation

Radhika Dua, Young Joon, Kwon, Siddhant Dogra, Daniel Freedman, Diana Ruan, Motaz Nashawaty, Danielle Rigau, Daniel Alexander Alber, Kang Zhang, Kyunghyun Cho, Eric Karl Oermann

机构 * New York University（纽约大学） ； NYU Langone Health（纽约大学兰格医学中心） ； Genentech（基因泰克） ； NYU Grossman School of Medicine（纽约大学格罗斯曼医学院） ； Wenzhou Medical University（温州医科大学） ； Macau University of Science and Technology（澳门科学大学） 纠错

专题命中 推理评测 ：planning(abstract)；分类 cs.CL、cs.AI、cs.LG

Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework

Jialin Li, Jinzhe Li, Gengxu Li, Yi Chang, Yuan Wu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Leveraging Vision-Language Models for Visual Grounding and Analysis of Automotive UI

Benjamin Raphael Ernhofer, Daniil Prokhorov, Jannica Langner, Dominik Bollmann

机构 * SPARKS Solutions GmbH（SPARKS解决方案有限公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

AgentSME for Simulating Diverse Communication Modes in Smart Education

Wen-Xi Yang, Tian-Fang Zhao

机构 * Guangdong Institute of Smart Education Jinan University Guangzhou, China ； School of Journalism \& Communication Jinan University Guangzhou, China 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

ContractEval: Benchmarking LLMs for Clause-Level Legal Risk Identification in Commercial Contracts

Shuang Liu, Zelong Li, Ruoyun Ma, Haiyan Zhao, Mengnan Du

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Rutgers University（罗格斯大学） ； Stanford University（斯坦福大学） ； New Jersey Institute of Technology（新泽西理工学院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs

Fangyi Yu

机构 * Fangyi Yu 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

MedBLINK: Probing Basic Perception in Multimodal Language Models for Medicine

Mahtab Bigverdi, Wisdom Ikezogwo, Kevin Zhang, Hyewon Jeong, Mingyu Lu, Sungjae Cho, Linda Shapiro, Ranjay Krishna

机构 * University of Washington（华盛顿大学） ； Massachusetts Institute of Technology（麻省理工学院） ； Seoul National University（首尔国立大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

The Architecture of Trust: A Framework for AI-Augmented Real Estate Valuation in the Era of Structured Data

Petteri Teikari, Mike Jarrell, Maryam Azh, Harri Pesola

机构 * Mill Hill Garage ； JB Real Estate Valuation & Advisory, LLC（JB Real Estate Valuation & Advisory LLC） ； Atlas Insights 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 46 pages, 6 figures

Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Study

Nandana Mihindukulasooriya, Niharika S. D'Souza, Faisal Chowdhury, Horst Samulowitz

机构 * IBM Research（IBM研究院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Accepted at LLM+Graph WS at VLDB 2025. 21 pages, 7 figures, 8 tables

GEMA-Score: Granular Explainable Multi-Agent Scoring Framework for Radiology Report Evaluation

Zhenxuan Zhang, Kinhei Lee, Peiyuan Jing, Weihang Deng, Huichi Zhou, Zihao Jin, Jiahao Huang, Zhifan Gao, Dominic C Marshall, Yingying Fang, Guang Yang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL