Recent advances in LLMs have demonstrated strong reasoning capabilities, largely stimulated by RLVR. However, the exploration of existing RLVR algorithms remains largely constrained by the knowledge and reasoning strategies already accessible to the policy model. Although recent methods introduce external expert demonstrations to broaden exploration, they typically rely on complete trajectory-level guidance, which can be sample-inefficient, information-sparse, and insufficiently adaptive to intermediate reasoning bottlenecks. Inspired by collaborative multi-agent systems, we propose AIPO, an enhanced reinforcement learning framework that improves LLM reasoning through active multi-agent interaction during exploration. Specifically, when encountering reasoning bottlenecks, AIPO enables the policy model to proactively consult three functional collaborative agents, namely the verify agent, knowledge agent, and reasoning agent, thereby obtaining fine-grained and state-dependent guidance during rollout. The resulting mixed-policy trajectories expose the policy to reasoning directions that may be difficult to discover through isolated on-policy exploration. To learn effectively from collaborator-provided tokens, we further introduce a corrected importance sampling coefficient together with a lower-bound clipping strategy to mitigate off-policy discrepancy and vanishing gradients. After training, the policy model reasons independently without relying on collaborative agents. Extensive experiments across mathematical, scientific, coding, and puzzle reasoning benchmarks show that AIPO consistently improves reasoning performance and generalizes across different policy models, collaborator backbones, and RLVR algorithms.
From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?
TMLR Group, Department of Computer Science, Hong Kong Baptist University;Stanford University;Mila - Qu\'ebec AI Institute;Cooperative Medianet Innovation Center, Shanghai Jiao Tong University
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人机混合增强智能国家重点实验室、视觉信息与应用国家工程研究中心、人工智能与机器人研究所);Nankai University(南开大学);The Hong Kong University of Science and Technology(Guangzhou)(香港科技大学(广州));School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件学院)
Route, Communicate, and Reason: Gated Routing and Adaptive Depth for Efficient Multi-Agent Reasoning
Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi(印度理工学院德里分校雅迪人工智能学院);Department of Electrical Engineering, Indian Institute of Technology Delhi(印度理工学院德里分校电气工程系)
Active Prompting with Chain-of-Thought for Large Language Models
The Hong Kong University of Science and Technology(香港科技大学);University of Toronto(多伦多大学);The University of Hong Kong(香港大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation
Shanghai Jiao Tong University(上海交通大学);Northeastern University(东北大学);Xiamen University Malaysia(厦门大学马来西亚分校);The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);Xspark AI
PerspAct: Enhancing LLM Situated Collaboration Skills through Perspective Taking and Active Vision
University of Milan-Bicocca(米兰-比科卡大学);University of Naples Federico II(那不勒斯费德里科二世大学);Oversonic Robotics(Oversonic机器人公司);University of Essex(埃塞克斯大学);TUM School of Social Sciences and Technology(慕尼黑技术大学社会科学与技术学院)
NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering
University of Macau(澳门大学);University of Chinese Academy of Sciences(中国科学院大学);The Australian National University(澳大利亚国立大学);Juntendo University(静冈大学);Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Central South University(中南大学);Southeast University(东南大学);China Agricultural University(中国农业大学);Adelaide University(阿德莱德大学)
Native Active Perception as Reasoning for Omni-Modal Understanding
The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);Nanyang Technological University(南洋理工大学);Qwen Team, Alibaba Group(阿里巴巴集团Qwen团队)
RuleMem: Active Rule Memory for Long-Term Conversational Agents
Sun Yat-sen University(中山大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Shenzhen Institute of Computing Sciences(深圳计算科学研究院);The Hong Kong Polytechnic University(香港理工大学);Tsinghua University(清华大学)