PCGRL+：强化学习关卡生成器中的扩展性、控制与泛化
PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators
AI总结：
本文将PCGRL环境迁移至Jax实现GPU并行仿真以解决训练规模瓶颈，通过引入随机关卡尺寸和冻结关键图块提升设计师控制力，并验证了部分观察在分布外地图上具有更强的泛化能力。
摘要：
通过强化学习进行程序化内容生成（PCGRL）已被引入作为一种方法，通过该方法，仅基于一组作为关卡质量和关键特征代理的可计算指标，即可训练可控的设计智能体。尽管PCGRL为游戏设计师提供了一套独特的功能支持，但它受到训练RL智能体这一计算密集型过程的限制，并且迄今为止仅限于生成相对较小的关卡。为了解决这一规模问题，我们在Jax中实现了多个PCGRL环境，使得学习和仿真的所有方面都在GPU上并行进行，从而实现更快的环境仿真；消除了RL训练期间CPU-GPU信息传输的瓶颈；并最终显著提升了训练速度。我们在这一新框架中复现了先前工作的若干关键结果，让模型训练时间比以往研究长得多，并在10亿时间步后评估其行为。为了给人类设计师提供更大的控制权，我们引入了随机化关卡尺寸和冻结关键游戏图块的“精确点”作为对抗过拟合的进一步方法。为了测试所学生成器的泛化能力，我们在大型、分布外的地图尺寸上评估模型，并发现部分观察尺寸能学习到更稳健的设计策略。
英文摘要：
Procedural Content Generation via Reinforcement Learning (PCGRL) has been introduced as a means by which controllable designer agents can be trained based only on a set of computable metrics acting as a proxy for the level's quality and key characteristics. While PCGRL offers a unique set of affordances for game designers, it is constrained by the compute-intensive process of training RL agents, and has so far been limited to generating relatively small levels. To address this issue of scale, we implement several PCGRL environments in Jax so that all aspects of learning and simulation happen in parallel on the GPU, resulting in faster environment simulation; removing the CPU-GPU transfer of information bottleneck during RL training; and ultimately resulting in significantly improved training speed. We replicate several key results from prior works in this new framework, letting models train for much longer than previously studied, and evaluating their behavior after 1 billion timesteps. Aiming for greater control for human designers, we introduce randomized level sizes and frozen "pinpoints" of pivotal game tiles as further ways of countering overfitting. To test the generalization ability of learned generators, we evaluate models on large, out-of-distribution map sizes, and find that partial observation sizes learn more robust design strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
