ProRL Agent：多轮LLM代理的RL训练rollout-as-a-service
ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents
AI总结：
本文提出ProRL Agent，通过API服务提供完整的代理rollout生命周期，支持多样化的代理任务，在无根HPC环境中提供标准化可扩展的沙箱环境，通过软件工程、数学、STEM和编程任务的RL训练验证其有效性。
摘要：
多轮LLM代理在解决复杂交互任务中日益重要，强化学习（RL）是改进其长周期行为的关键。然而，RL训练需要生成大量沙箱rollout轨迹，现有基础设施常将rollout编排与训练循环耦合，使系统难以迁移和维护。基于rollout-as-a-service理念，我们提出ProRL Agent，一种可扩展的基础设施，通过API服务提供完整的代理rollout生命周期。ProRL Agent还提供标准化且可扩展的沙箱环境，支持在无根HPC设置中执行多样化的代理任务。我们通过在软件工程、数学、STEM和编程任务上的RL训练验证了ProRL Agent。ProRL Agent是开源的，并作为NVIDIA NeMo Gym的一部分集成。
英文摘要：
Multi-turn LLM agents are increasingly important for solving complex, interactive tasks, and reinforcement learning (RL) is a key ingredient for improving their long-horizon behavior. However, RL training requires generating large numbers of sandboxed rollout trajectories, and existing infrastructures often couple rollout orchestration with the training loop, making systems hard to migrate and maintain. Under the rollout-as-a-service philosophy, we present ProRL Agent , a scalable infrastructure that serves the full agentic rollout lifecycle through an API service. ProRL Agent also provides standardized and extensible sandbox environments that support diverse agentic tasks in rootless HPC settings. We validate ProRL Agent through RL training on software engineering, math, STEM, and coding tasks. ProRL Agent is open-sourced and integrated as part of NVIDIA NeMo Gym.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
