发表机构
Amazon AGI（亚马逊AGI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
AutoGym提出蓝图优先的框架，从领域种子或轨迹生成可验证的智能体环境，通过显式参数和主动课程合成实现细粒度难度调控，生成挑战前沿模型的任务。
AI 中文摘要
使用强化学习训练智能体需要一个环境（gym），该环境包含一个任务、一个可执行的环境（任务可在其中尝试）以及一个能够可靠区分成功与失败的验证器。构建此类环境目前仍是手动、昂贵且静态的过程。随着模型能力的提升，任务集逐渐饱和，并日益受到数据污染的影响。合成生成提供了规模化的解决方案，但单次合成产生的任务，其难度在很大程度上只是表面性的。能力相当的模型即使面对措辞复杂的任务也能解决，而正确性必须由不可靠的LLM评判者事后裁定。我们提出了AutoGym，一个从最小领域种子或先前的模型轨迹生成完整环境（任务、可执行环境和验证器）的框架。AutoGym引入了三种机制。（1）蓝图优先生成：在环境具体化之前指定有效的解空间、环境要求和验证标准，使可解性成为构建的前提条件，而非事后验证的属性。（2）显式生成参数控制任务拓扑、交互深度、能力轴、问题混淆和干扰项构成，实现细粒度的难度调控。（3）主动课程合成使用基于性能的校准，随着模型能力的演变调整这些参数的分布。在生产力和时间推理场景中，AutoGym生成覆盖能力谱系的环境，包括挑战前沿模型的实例。
英文摘要
Training agents with reinforcement learning requires a gym, comprising a task, an executable environment in which the task can be attempted, and a verifier that reliably distinguishes success from failure. Constructing such gyms remains manual, expensive, and static. Task sets saturate as models improve and are increasingly exposed to contamination. Synthetic generation offers scale, but single-pass synthesis produces tasks whose difficulty is largely cosmetic. Models comparable in capability solve them despite convoluted phrasing, and correctness must be adjudicated post-hoc by unreliable LLM judges. We present AutoGym, a framework that generates complete gyms (tasks, executable environments, and verifiers) from a minimal domain seed or prior model trajectories. AutoGym introduces three mechanisms. (1) Blueprint-first generation specifies the valid solution space, environment requirements, and verification criteria before the environment is materialized, making solvability a construction prerequisite rather than a property verified after the fact. (2) Explicit generation parameters control task topology, interaction depth, capability axes, question obfuscation, and distractor composition, enabling fine-grained difficulty steering. (3) Active curriculum synthesis uses performance-informed calibration to adjust the distribution over these parameters as model capabilities evolve. Across productivity and temporal-reasoning settings, AutoGym generates gyms spanning the capability spectrum, including instances that challenge frontier models.
CommentsSubmitted to NeurIPS 2026 Workshop: Who Verifies the Agents?
