发表机构
Hunyuan Team, Tencent（腾讯混元团队）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对现有协同演化方法的不足，提出环境演化方法，通过off-policy逐代提升环境难度，在Terminal-Bench 2.1上显著提升Qwen系列模型性能。
AI 中文摘要
扩展可交互且可验证的环境对于训练终端智能体至关重要。随着前沿模型能力增强，从零合成的环境挑战性降低，提供的学习信号有限。近期的协同演化方法基于rollout过程中暴露的弱点，迭代合成接近模型可学习前沿的环境，但它们依赖on-policy rollout，限制了泛化能力，且当模型变强时无法持续提供学习信号。本文提出环境演化方法，该方法off-policy逐步提升环境难度，在训练过程中逐代调度演化后的环境以提供持续学习信号。我们从多轮学习目标中推导出影响环境难度的三个演化方向，随后通过精心设计的多智能体框架沿这些方向实现演化。与Hy4 preview、Claude Opus 5和GPT-5.6 Sol的定量rollout实验表明，环境演化始终能生成更具挑战性的环境。我们通过简单的长 horizon RL训练，在Qwen3.6-27B和Qwen3.6-35B-A3B上验证了其有效性，使它们在Terminal-Bench 2.1上的性能分别提升14.4和18.0个百分点。
英文摘要
Scaling interactive and verifiable environments is critical for training terminal agents. As frontier models become more capable, environments synthesized from scratch become less challenging and thus provide limited learning signals. Recent co-evolution methods iteratively synthesize environments near the model's learnable frontier based on weaknesses exposed during rollouts. However, their dependence on on-policy rollouts limits generalization and the continuous provision of learning signals as the model becomes stronger. In this paper, we propose environment evolution, which incrementally increases environment difficulty off-policy and schedules the evolved environments generation by generation during training to provide continuous learning signals. We derive three evolution directions that influence environment difficulty from the multi-turn learning objective and then implement evolution along these directions through a loop-engineered multi-agent harness. Quantitative rollout experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol show that environment evolution consistently produces more difficult environments. We validate its effectiveness on Qwen3.6-27B and Qwen3.6-35B-A3B through simple long-horizon RL training, improving their performance by 14.4 and 18.0 percentage points on Terminal-Bench 2.1, respectively.
