大模型推理能力 - arXivDaily 专题

2510.02172 2025-10-03 cs.CL 70%

RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalization

Zhaoning Yu, Will Su, Leitian Tao, Haozhu Wang, Aashu Singh, Hanchao Yu, Jianyu Wang, Hongyang Gao, Weizhe Yuan, Jason Weston, Ping Yu, Jing Xu

机构 * Iowa State University（爱荷华州立大学）； FAIR at Meta SuperIntelligence Lab（Meta超智能实验室FAIR团队）； Meta

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

2503.22678 2025-10-02 cs.CL 70%

Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions

Mohammad Almansoori, Komal Kumar, Hisham Cholakkal

机构 * Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed人工智能大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 14 page, 4 figures, 61 references, presented in MICCAI (Oral)

2509.25281 2025-10-01 cond-mat.mtrl-sci cs.LG 70%

Mechanisms of Matter: Language Inferential Benchmark on Physicochemical Hypothesis in Materials Synthesis

Yingming Pu, Tao Lin, Hongyu Chen

机构 * Westlake University（西湖大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.LG

2508.04361 2025-09-30 cs.AI 70%

OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing

Fuqing Bie, Shiyu Huang, Xijia Tao, Zhiqin Fang, Leyi Pan, Junzhe Chen, Min Ren, Liuyu Xiang, Zhaofeng He

机构 * Beijing University of Posts and Telecommunications（北京邮电大学）； XPENG ； The University of Hong Kong（香港大学）； Tsinghua University（清华大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2501.05414 2025-09-30 cs.CL 70%

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation

Xi Ye, Fangcong Yin, Yinghui He, Joie Zhang, Howard Yen, Tianyu Gao, Greg Durrett, Danqi Chen

专题命中推理评测：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments COLM 2025. Data and code available at: https://princeton-pli.github.io/LongProc

2509.21451 2025-09-29 cs.CV cs.CL 70%

VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding

Abdul Waheed, Zhen Wu, Dareen Alharthi, Seungone Kim, Bhiksha Raj

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Work in progress

2503.23768 2025-09-29 cs.CL cs.CV 70%

Texture or Semantics? Vision-Language Models Get Lost in Font Recognition

Zhecheng Li, Guoxian Song, Yujun Cai, Zhen Xiong, Junsong Yuan, Yiwei Wang

机构 * University of California, San Diego（加州大学圣地亚哥分校）； ByteDance（字节跳动）； The University of Queensland（昆士兰大学）； University of Southern California（南加州大学）； University at Buffalo（布法罗大学）； University of California, Merced（加州大学默塞德分校）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments Accepted to COLM 2025

2503.03064 2025-09-29 cs.CL 70%

Improving LLM-as-a-Judge Inference with the Judgment Distribution

Victor Wang, Michael J. Q. Zhang, Eunsol Choi

机构 * Department of Computer Science（计算机科学系）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）； New York University（纽约大学）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

2506.01047 2025-09-26 cs.CL 70%

CHEER-Ekman: Fine-grained Embodied Emotion Classification

Phan Anh Duong, Cat Luong, Divyesh Bommana, Tianyu Jiang

机构 * University of Cincinnati（辛辛那提大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments ACL 2025

2509.19274 2025-09-24 cs.CL cs.MM 70%

DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture

Arijit Maji, Raghvendra Kumar, Akash Ghosh, Anushka, Nemil Shah, Abhilekh Borah, Vanshika Shah, Nishant Mishra, Sriparna Saha

机构 * Indian Institute of Technology Patna（印度理工学院帕纳巴分校）； Banasthali Vidyapeeth University（班纳萨利大学）； Pandit Deendayal Energy University（德英德能源大学）； Manipal University Jaipur（马哈拉施特拉邦大学贾伊普尔分校）； Dwarkadas J. Sanghvi College of Engineering（德瓦尔卡斯J.桑格维工程学院）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments EMNLP MAINS 2025

2509.17740 2025-09-23 cs.CV cs.CL 70%

WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification

Yiwen Jiang, Deval Mehta, Siyuan Yan, Yaling Shen, Zimu Wang, Zongyuan Ge

机构 * Faculty of Engineering, Monash University（墨尔本大学工程学院）； AIM for Health Lab, Faculty of IT, Monash University（墨尔本大学信息技术学院健康人工智能实验室）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2025 (Main)

2406.17974 2025-09-23 cs.CL cs.CV 70%

Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts

Xuyang Wu, Yuan Wang, Hsin-Tai Wu, Zhiqiang Tao, Yi Fang

机构 * Santa Clara University（圣克拉拉大学）； DOCOMO Innovations, Inc.（DOCOMO创新公司）； Rochester Institute of Technology（罗切斯特理工学院）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

Comments EMNLP Findings

2509.15366 2025-09-22 cs.AI 70%

Diagnostics of cognitive failures in multi-agent expert systems using dynamic evaluation protocols and subsequent mutation of the processing context

Andrejs Sorstkins, Josh Bailey, Dr Alistair Baron

机构 * School of Computing and Communications（计算与通讯学院）； Lancaster University（兰卡斯特大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

Comments Dissertation and research project created in collaboration with JobFair LTD

2509.13244 2025-09-17 cs.CL 70%

Evaluating LLM Alignment on Personality Inference from Real-World Interview Data

Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G. Coifman, Ruoming Jin

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 8 pages, 3 figures

2509.10345 2025-09-16 cs.CV cs.AI 70%

Towards Understanding Visual Grounding in Visual Language Models

Georgios Pantazopoulos, Eda B. Özyiğit

机构 * The Alan Turing Institute（艾伦·图灵研究所）； Heriot-Watt University（赫瑞-沃德大学）

专题命中推理评测：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.AI

2508.03700 2025-09-12 cs.HC cs.AI 70%

MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning

Liujian Tang, Shaokang Dong, Yijia Huang, Minqi Xiang, Hongtao Ruan, Bin Wang, Shuo Li, Zhiheng Xi, Zhihui Cao, Hailiang Pang, Heng Kong, He Yang, Mingxu Chai, Zhilin Gao, Xingyu Liu, Yingnan Fu, Jiaming Liu, Xuanjing Huang, Yu-Gang Jiang, Tao Gui, Qi Zhang, Kang Wang, Yunke Zhang, Yuran Wang

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2509.05315 2025-09-09 cs.RO cs.AI cs.CV 70%

Evaluation of Large Language Models for Anomaly Detection in Autonomous Vehicles

Petros Loukas, David Bassir, Savvas Chatzichristofis, Angelos Amanatiadis

机构 * Democritus University of Thrace（德米特里乌斯大学）； Dongguan University of Technology（东莞理工大学）； ENS-Paris-Saclay University（巴黎-萨克雷大学）； Neapolis University Pafos（纳皮奥斯大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2509.03565 2025-09-05 cs.CL cs.MM 70%

ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference

Qi Chen, Jingxuan Wei, Zhuoya Yao, Haiguang Wang, Gaowei Wu, Bihui Yu, Siyuan Li, Cheng Tan

机构 * University of Chinese Academy of Sciences（中国科学院大学）； Zhejiang University（浙江大学）； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.CL

Comments Accepted to ACM MM 2025

2509.02198 2025-09-03 cs.CL 70%

FActBench: A Benchmark for Fine-grained Automatic Evaluation of LLM-Generated Text in the Medical Domain

Anum Afzal, Juraj Vladika, Florian Matthes

机构 * Technical University of Munich（慕尼黑技术大学）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

2412.14446 2025-09-03 cs.CV cs.LG 70%

VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision

Yi Xu, Yuxin Hu, Zaiwei Zhang, Gregory P. Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M. Wolff, Xin Huang

机构 * Cruise LLC (GM)（Cruise LLC（GM））； Northeastern University（东北大学）； OpenAI（开放人工智能研究所）； Meta ； University of Washington（华盛顿大学）； Waymo LLC

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.LG

Comments Accepted by CoRL 2025

2508.19484 2025-08-28 cs.CL 70%

Rule Synergy Analysis using LLMs: State of the Art and Implications

Bahar Bateni, Benjamin Pratt, Jim Whitehead

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Submitted for publication at the IEEE Transactions on Games 2024, Special Issue on Large Language Models and Games (10 pages excluding appendix, 3 figures)

2508.19432 2025-08-28 cs.AI 70%

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

Yao Fu, Xianxuan Long, Runchao Li, Haotian Yu, Mu Sheng, Xiaotian Han, Yu Yin, Pan Li

机构 * Case Western Reserve University（凯斯西储大学）； Hangzhou Dianzi University（杭州电子科技大学）

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

Comments Accepted to EMNLP2025 main conference (poster)

2508.16987 2025-08-26 cs.AI cs.CV 70%

WebSight: A Vision-First Architecture for Robust Web Agents

Tanvir Bhathal, Asanshay Gupta

机构 * Stanford University（斯坦福大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

2508.13930 2025-08-20 cs.IR cs.AI 70%

InPars+: Supercharging Synthetic Data Generation for Information Retrieval Systems

Matey Krastev, Miklos Hamar, Danilo Toapanta, Jesse Brouwers, Yibin Lei

机构 * University of Amsterdam（阿姆斯特丹大学）

专题命中推理评测：chain-of-thought(abstract)；CoT(abstract)；分类 cs.AI

2508.10014 2025-08-15 cs.CL 70%

PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?

Lingfeng Zhou, Jialing Zhang, Jin Gao, Mohan Jiang, Dequan Wang

专题命中推理评测：reasoning(abstract)；test-time compute(abstract)；分类 cs.CL

Comments Accepted by COLM 2025

2502.17810 2025-08-12 cs.CL eess.AS 70%

URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models

Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, Xie Chen

专题命中推理评测：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

2508.06600 2025-08-12 cs.CL cs.IR 70%

BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin

机构 * University of Waterloo（滑铁卢大学）； CSIRO（澳大利亚联邦科学与工业研究组织）； Independent（独立研究者）； Carnegie Mellon University（卡内基梅隆大学）； The University of Queensland（昆士兰大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.CL

2508.03396 2025-08-06 cs.AI 70%

Hide and Seek with LLMs: An Adversarial Game for Sneaky Error Generation and Self-Improving Diagnosis

Rui Zou, Mengqi Wei, Yutao Zhu, Jirong Wen, Xin Zhao, Jing Chen

专题命中推理评测：reasoning(abstract)；math reasoning(abstract)；分类 cs.AI

2505.17827 2025-08-05 cs.CL 70%

Not All Tokens Are What You Need In Thinking

Hang Yuan, Bin Yu, Haotian Li, Shijun Yang, Christina Dan Wang, Zhou Yu, Xueyin Xu, Weizhen Qi, Kai Chen

专题命中推理评测：reasoning(abstract)；CoT(abstract)；分类 cs.CL

Comments 11 pages, 7 figures and 3 tables

2507.21585 2025-07-30 cs.AI 70%

SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation

Hao Ye, Mengshi Qi, Zhaohong Liu, Liang Liu, Huadong Ma

机构 * State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，北京邮电大学）

专题命中推理评测：reasoning(abstract)；planning(abstract)；分类 cs.AI

1. 推理评测 10530 篇

RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalization

Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions

Mechanisms of Matter: Language Inferential Benchmark on Physicochemical Hypothesis in Materials Synthesis

OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation

VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding

Texture or Semantics? Vision-Language Models Get Lost in Font Recognition

Improving LLM-as-a-Judge Inference with the Judgment Distribution

CHEER-Ekman: Fine-grained Embodied Emotion Classification

DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture

WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification

Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts

Diagnostics of cognitive failures in multi-agent expert systems using dynamic evaluation protocols and subsequent mutation of the processing context

Evaluating LLM Alignment on Personality Inference from Real-World Interview Data

Towards Understanding Visual Grounding in Visual Language Models

MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning

Evaluation of Large Language Models for Anomaly Detection in Autonomous Vehicles

ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference

FActBench: A Benchmark for Fine-grained Automatic Evaluation of LLM-Generated Text in the Medical Domain

VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision

Rule Synergy Analysis using LLMs: State of the Art and Implications

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

WebSight: A Vision-First Architecture for Robust Web Agents

InPars+: Supercharging Synthetic Data Generation for Information Retrieval Systems

PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?

URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models

BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Hide and Seek with LLMs: An Adversarial Game for Sneaky Error Generation and Self-Improving Diagnosis

Not All Tokens Are What You Need In Thinking

SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation