发表机构
Eindhoven University of Technology（埃因霍温理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Factoriax是一个基于JAX的GPU加速工厂建造模拟器，用于强化学习；其Easy Rocket基准测试中，PPO智能体通过课程奖励学习资源收集与机器建造，但未能完成最终任务。
AI 中文摘要
我们介绍了Factoriax，一个用JAX编写的GPU加速工厂建造模拟器。在Factoriax中，智能体必须收集资源，使用这些资源建造机器，然后通过在地图上布置机器来构建生产流水线，从而实现收集和制作过程的自动化。本文讨论了Factoriax模拟器的结构，以及一个名为Easy Rocket的初始基准测试，在该基准测试中，智能体的任务是在有限的游戏刻数内建造一个名为Rocket的资源密集型机器以逃离星球。我们还发布了在Easy Rocket上进行的多次基于PPO的训练运行的结果。Factoriax的设计目标是快速。一次10亿步的PPO训练运行，相当于500,000个回合，在单个NVIDIA A100上运行Easy Rocket大约需要8分钟。一台标准的笔记本电脑GPU可以在大约84分钟内完成同样的运行。我们训练出的PPO智能体通过一个与手动设计的成就集合直接相关的课程奖励，学会了收集资源、用这些资源制作机器，并将它们放置在地图上。训练结束后，智能体未能以功能性的空间配置放置机器，未能完全完成基准测试，这一挑战留待未来的尝试。
英文摘要
We introduce Factoriax, a GPU-accelerated factory-building simulator written in JAX. In Factoriax, an agent must collect resources, build machines using those resources, and then automate the collection and crafting process by arranging machines on the map to build production pipelines. This paper discusses the structure of the Factoriax simulator and an initial benchmark called Easy Rocket in which an agent is tasked with building a resource-intensive machine called the Rocket in a limited number of game ticks to escape the planet. We also publish results from a number of PPO-based training runs on Easy Rocket. Factoriax is built to be fast. A 1-billion-step PPO training run, equivalent to 500,000 episodes, runs on Easy Rocket in about 8 minutes on a single NVIDIA A100. A standard laptop GPU can complete the same run in about 84 minutes. Our trained PPO agent learns to gather resources, craft machines from those resources, and place them on the map through a curriculum reward directly tied to a manually designed set of achievements. After training, the agent does not place machines in a functional spatial configuration, failing to fully complete the benchmark, and leaving the challenge open for future attempts.
