Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration
The Chinese University of Hong Kong(香港中文大学);LLM Department, Tencent(腾讯语言模型部门);Mohamed bin Zayed University of Artificial Intelligence(马尔代夫人工智能大学);McGill University(麦吉尔大学);City University of Hong Kong(香港城市大学);The University of Edinburgh(爱丁堡大学)
Search-integrated reasoning enables language agents to transcend static parametric knowledge by actively querying external sources. However, training these agents via reinforcement learning is hindered by the multi-scale credit assignment problem: existing methods typically rely on sparse, trajectory-level rewards that fail to distinguish between high-quality reasoning and fortuitous guesses, leading to redundant or misleading search behaviors. To address this, we propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention, with both components jointly optimized during training. Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories, and a Meta-Refiner, which selectively diagnoses and repairs flawed steps via a 'cut-and-regenerate' mechanism. To provide fine-grained supervision, we introduce a hybrid reward design that couples outcome correctness with a dense process reward quantifying the information density of retrieved evidence. Theoretically, we formalize the Actor-Refiner interaction as a smoothed mixture policy, proving that selective correction yields strict performance gains over strong baselines. Extensive experiments across various general and multi-hop QA datasets demonstrate that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales, achieving superior reasoning accuracy with minimal overhead.
Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Driving
Tianjin Key Laboratory of Intelligent Unmanned Swarm Technology and System, School of Electrical and Information Engineering, Tianjin University(天津智能无人群技术与系统重点实验室,电气与信息工程学院,天津大学);Key Laboratory of System Control and Information Processing, Ministry of Education of China(系统控制与信息处理重点实验室,中华人民共和国教育部);School of Mechanical Engineering, Shanghai Jiao Tong University(机械工程学院,上海交通大学)
Federated Multi-Agent Actor-Critic Learning for Age Sensitive Mobile Edge Computing
Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tsinghua University(清华大学);Hong Kong University of Science and Technology(香港科技大学);Pengcheng Lab(鹏城实验室)
Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approach
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);SenseTime Research(商汤科技研究院);Shanghai Jiao Tong University(上海交通大学);The University of Manchester(曼彻斯特大学)
Efficient Policy Generation in Multi-Agent Systems via Hypergraph Neural Network
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
A Visual Communication Map for Multi-Agent Deep Reinforcement Learning
Institute for Intelligent Systems Research and Innovation, Deakin University(迪肯大学智能系统研究与创新研究所);School of Information Technology, Deakin University(迪肯大学信息技术学院)
Multi-Agent Collaboration Mechanisms: A Survey of LLMs
School of Computer Science and Information Technology, University College Cork(科克大学学院计算机科学与信息技术学院);Department of Information Convergence Engineering, Pusan National University(釜山国立大学信息融合工程系);School of Computer Science and Statistics, Trinity College Dublin(都柏林圣三一学院计算机科学与统计学院)
OMAC: A Holistic Optimization Framework for LLM-Based Multi-Agent Collaboration
Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, United States(得克萨斯大学奥斯汀分校电子与计算机工程系);Intuit AI Research, Mountain View, United States(Intuit AI研究)
Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport
Department of Electrical Engineering, the University of Texas at Arlington(电气工程系,德克萨斯大学阿灵顿分校);Department of Mechanical Engineering, Carnegie Mellon University(机械工程系,卡内基梅隆大学)