扩散模型是实时游戏引擎
Diffusion Models Are Real-Time Game Engines
AI总结:
GameNGen是首个由神经模型驱动的实时游戏引擎,在DOOM上以20 FPS运行,通过扩散模型实现高保真交互式模拟,性能媲美JPEG压缩。
摘要:
我们提出了GameNGen,这是首个完全由神经模型驱动的游戏引擎,它还能以高质量实现与复杂环境的实时交互,支持长时间轨迹的交互式模拟。在经典游戏DOOM上训练时,GameNGen提取游戏玩法,并利用它生成一个可玩的交互式环境,能够模拟新的轨迹。GameNGen在单个TPU上以每秒20帧的速度运行,并在持续数分钟的多玩家会话中保持稳定。下一帧预测的PSNR达到29.4,与有损JPEG压缩相当。人类评估者区分游戏短片与模拟短片的能力仅略优于随机猜测,即使在5分钟的自回归生成之后也是如此。GameNGen的训练分为两个阶段:(1)强化学习智能体学习玩游戏并记录训练会话,(2)训练扩散模型根据过去的帧和动作序列生成下一帧。条件增强有助于确保长时间轨迹上的稳定自回归生成,解码器微调则提高了视觉细节和文本的保真度。
英文摘要:
We present GameNGen, the first game engine powered entirely by a neural model that also enables real-time interaction with a complex environment over long trajectories at high quality. When trained on the classic game DOOM, GameNGen extracts gameplay and uses it to generate a playable environment that can interactively simulate new trajectories. GameNGen runs at 20 frames per second on a single TPU and remains stable over extended multi-minute play sessions. Next frame prediction achieves a PSNR of 29.4, comparable to lossy JPEG compression. Human raters are only slightly better than random chance at distinguishing short clips of the game from clips of the simulation, even after 5 minutes of auto-regressive generation. GameNGen is trained in two phases: (1) an RL-agent learns to play the game and the training sessions are recorded, and (2) a diffusion model is trained to produce the next frame, conditioned on the sequence of past frames and actions. Conditioning augmentations help ensure stable auto-regressive generation over long trajectories, and decoder fine-tuning improves the fidelity of visual details and text.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。