Reward-guided search methods have demonstrated strong potential in enhancing tool-using agents by effectively guiding sampling and exploration over complex action spaces. As a core design, those search methods utilize process reward models (PRMs) to provide step-level rewards, enabling more fine-grained monitoring. However, there is a lack of systematic and reliable evaluation benchmarks for PRMs in tool-using settings. In this paper, we introduce ToolPRMBench, a large-scale benchmark specifically designed to evaluate PRMs for tool-using agents. ToolPRMBench is built on top of several representative tool-using benchmarks and converts agent trajectories into step-level test cases. Each case contains the interaction history, a correct action, a plausible but incorrect alternative, and relevant tool metadata. We respectively utilize offline sampling to isolate local single-step errors and online sampling to capture realistic multi-step failures from full agent rollouts. A multi-LLM verification pipeline is proposed to reduce label noise and ensure data quality. We conduct extensive experiments across large language models, general PRMs, and tool-specialized PRMs on ToolPRMBench. The results reveal clear differences in PRM effectiveness and highlight the potential of specialized PRMs for tool-using. Code and data will be released at https://github.com/David-Li0406/ToolPRMBench.
GFRIEND: Generative Few-shot Reward Inference through EfficieNt DPO
College of Computing and Data Science,Nanyang Technological University (NTU), Singapore(南洋理工大学计算与数据科学学院);Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly(LILY),NTU,Singapore(南洋理工大学与UBC联合卓越老年活跃生活研究中心(LILY))
Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach
AImotion Bavaria, Technische Hochschule Ingolstadt(巴伐利亚AImotion、英格尔施泰特技术大学);School of Computation, Information and Technology, TU Munich(计算、信息与技术学院,慕尼黑技术大学)
Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment
University of Science and Technology of China (USTC)(中国科学技术大学);Tencent Games(腾讯游戏);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Suzhou Institute for Advanced Research, USTC(中国科学技术大学苏州研究院)
RLPR: Extrapolating RLVR to General Domains without Verifiers
Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Shanghai Qi Zhi Institute(上海启智研究院);Harbin Institute of Technology(哈尔滨工业大学);Beijing University of Posts and Telecommunications(北京邮电大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Reinforcement Learning in GUI Agents: A Survey Toward Digital Inhabitants
Shandong University(山东大学);The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University(香港大学);Shanghai Jiao Tong University(上海交通大学);Tencent(腾讯)
Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);iFLYTEK AI Research(iFLYTEK人工智能研究院);Hong Kong University of Science and Technology(香港科学与技术大学)
Imitation Learning from Observation with Automatic Discount Scheduling
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);Shanghai Qi Zhi Institute(上海期智研究院);UC Berkeley(加州大学伯克利分校);Washington University in St. Louis(圣路易斯华盛顿大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing
Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University(浙江大学);Westlake University(西湖大学);Nanjing University(南京大学);University of Southern California(南加州大学)
DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
Stanford University(斯坦福大学);University of Tokyo(东京大学);RIKEN AIP(日本理化学研究所革新智能研究中心);UC Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Amazon AWS(亚马逊云科技)
Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China(计算与智能研究所,哈尔滨工业大学,深圳,中国);Peng Cheng Laboratory, Shenzhen, China(鹏城实验室,深圳,中国)
Advancing LLM Reasoning Generalists with Preference Trees
Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Northeastern University(东北大学);Renmin University of China(中国人民大学);BUPT(北京邮电大学);Tencent(腾讯)
VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Models
Zhejiang University(浙江大学);Meituan Group(美团集团);Peking University(北京大学);University of Electronic Science and Technology of China(电子科技大学);The Hong Kong University of Science and Technology(香港科技大学)
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);Soochow University(苏州大学);Stony Brook University(石溪大学);The Chinese University of Hong Kong(香港中文大学)