arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.04484cs.AIcs.LGcs.MA
收藏

AgentJet:一种用于智能体强化学习的灵活群体训练框架

AgentJet: A Distributed Swarm Training Framework for Agentic Reinforcement Learning

Tongyi Lab, Alibaba Group(通义实验室,阿里巴巴集团)

Qingxu Fu, Boyin Liu, Shuchang Tao, Zhaoyang Liu, Cheng Chen, Xuanfa Jin, Rong Zhu, Bolin Ding

AI总结:

提出AgentJet,一种解耦的多节点群体训练框架,支持异构多模型强化学习、多任务鸡尾酒训练、容错执行和实时代码迭代,并通过上下文跟踪模块实现1.5-10倍训练加速。

摘要:

我们提出了AgentJet,一个用于大型语言模型(LLM)智能体强化学习的分布式群体训练框架。与将智能体运行与模型优化紧密耦合的集中式框架不同,AgentJet采用解耦的多节点架构,其中群体服务器节点托管可训练模型并在GPU集群上运行优化,而群体客户端节点在任意设备上执行任意智能体。这种设计提供了集中式框架难以支持的能力:(1)异构多模型强化学习,支持训练具有多个LLM作为大脑的异构多智能体团队;(2)具有隔离智能体运行时的多任务鸡尾酒训练;(3)容错执行,防止外部环境故障中断训练过程;(4)实时代码迭代,允许通过替换群体客户端节点在训练期间编辑智能体。为了支持多模型、多轮和多智能体设置中的高效强化学习,AgentJet引入了一个带有时间线合并的上下文跟踪模块,该模块合并冗余上下文并实现1.5-10倍的训练加速。最后,AgentJet引入了一个自动化研究系统,该系统以研究主题为输入,并在大规模集群上自主进行长期、多天的强化学习研究。通过利用群体架构,该系统在无需人工干预的情况下复现了强化学习研究人员的关键探索工作流程。

英文摘要:

Training reinforcement learning (RL) policies for large language model (LLM) agents requires optimizing multi-turn trajectories that interact with external environments. Existing training frameworks struggle with runtime failures, single-model constraints, incompatible task environments, and redundant context. We present AgentJet, a distributed swarm training framework based on a decoupled multi-node architecture. AgentJet treats the server--client topology as configurable: swarm servers host trainable models and perform optimization on GPU clusters, while detachable swarm clients execute arbitrary agents and communicate through OpenAI-compatible APIs. Reconfiguring this topology supports heterogeneous multi-model RL, mixed-task training with isolated runtimes, fault-tolerant execution, and live code iteration through hot-swappable clients. AgentJet also introduces context tracking with timeline merging, reducing actor-update time by 6.25x on AppWorld. The same detachable-client design supports an automated research system that conducts long-horizon, multi-day RL studies on large-scale clusters with limited human intervention. AgentJet is open-source and compatible with agent systems that issue standard LLM inference requests.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑