Player-1：从合成经验中学习游戏玩法
Player-1: Learning to Play from Synthetic Experience
AI总结：
本研究提出Player-1框架，通过合成经验学习游戏控制，扩展动作空间并构建合成环境，经PPO强化学习后，在MineDojo等任务上较基线与SFT策略实现显著性能提升。
摘要：
计算机使用智能体已展现出解读屏幕、执行键盘与鼠标操作的强大能力，但游戏凭借其视觉丰富的世界与多样的机制，仍是通用数字控制领域中极具挑战性的下一步目标。当前游戏智能体学习流程常依赖大规模演示，却难以覆盖动态游戏状态，且交互式训练的可扩展环境仍难以获取。我们提出Player-1，一种通过合成经验学习游戏控制的后训练框架，无需依赖演示。我们为计算机使用动作空间扩展了相对指针运动、持续输入与复合动作。为进行监督初始化，我们在6044条精心整理的游戏轨迹上微调计算机使用模型，这些轨迹将记录的动作与任务指令及视觉推理配对。我们构建了Player-1-Gym，共16个围绕游戏通用控制需求设计的合成2D与3D环境，其具备可程序化验证的奖励、随机化的观测与控制动态，支持使用PPO算法进行多环境强化学习。最终得到的Player-1-9B与Player-1-35B-A3B在MineDojo-300任务上的成功率，较其SFT策略分别提升27.8和17.7个百分点；Player-1-9B的成功率达38.33%，而评估中最强基线的成功率为12.63%，在ViZDoom-10与GameWorld任务上也较SFT有进一步提升。进一步分析显示，更广泛的合成环境覆盖可提升迁移能力，且强化学习过程中，结合多个键盘与鼠标输入的复合动作使用量会增加。我们证明，从合成经验中学习可显著改善超越监督初始化的游戏控制，其提升效果可泛化至强化学习训练套件之外。
英文摘要：
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
