arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.22316cs.LGcs.AIcs.MA
收藏

学习引导大语言模型智能体与多智能体系统

Learning to Steer LLM Agents and Multi-Agent Systems

National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Qwen Business Unit of Alibaba(阿里巴巴通义千问业务事业部)

Leyang Shen, Yaxin Gong, Yang Zhang, Chenxiao Fan, Jiakai Wang, Dong Wang, Yang Liu, Fuli Feng, Chun Kai Ling, Tat-Seng Chua

AI总结:

本研究针对LLM智能体及多智能体系统的目标漂移问题,提出引导智能体,结合GRPO与强化学习训练,可监督并干预执行,提升系统性能效率,且具备泛化、迁移及可控干预频率的优势。

摘要:

基于大语言模型(LLM)的智能体及多智能体系统正日益通过长期工具使用与智能体间协作解决复杂任务,但这类系统在执行过程中易偏离整体目标,浪费执行资源并导致不理想结果,该失败被称为目标漂移。本研究提出一种引导智能体,其可实时监督执行过程,在漂移发生时将系统拉回正轨。引导智能体观察执行上下文,决定何时以低成本快速路径介入,以及如何以高成本慢速路径介入。为解锁其引导智能——即评估目标漂移与生成有效干预的能力,我们结合成对排序目标与组相对策略优化(GRPO),通过强化学习联合训练两条路径。在信息检索搜索任务上的实验表明,该引导智能体可为单智能体与多智能体系统提供有效监督,提升其性能与效率;所学引导智能可泛化至域外任务、迁移至未见执行器,且无需重新训练即可支持可控干预频率。

英文摘要:

Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift. In this work, we propose a steering agent that supervises execution on the fly and steers the system back on track when drift occurs. The steering agent observes the execution context, deciding when to intervene with a low-cost fast path and how to intervene with a high-cost slow path. To unlock its steering intelligence, the ability to assess goal drift and generate effective interventions, we jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO). Experiments on information-seeking search tasks show that the steering agent provides effective supervision for both single-agent and multi-agent systems, improving their performance and efficiency. The learned steering intelligence generalizes to out-of-domain tasks, transfers to unseen executors, and supports controllable intervention frequency without retraining.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑