arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.13472cs.CVcs.AI
收藏

可提示游戏模型:通过掩码扩散模型实现文本引导的游戏模拟

Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models

University of Trento(特伦托大学);Snap Inc.(Snap 公司);LTCI, Télécom Paris, Institut Polytechnique de Paris(巴黎高等电信学院,巴黎综合理工学院);MPI for Informatics, SIC(马克斯·普朗克信息学研究所,SIC);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)

Willi Menapace, Aliaksandr Siarohin, Stéphane Lathuilière, Panos Achlioptas, Vladislav Golyanik, Sergey Tulyakov, Elisa Ricci

AI总结:

本文提出 PGM,通过自然语言动作与目标提示结合掩码扩散、动画模型和组合式 NeRF,实现语义可控的游戏模拟与导演模式,并发布 Tennis、Minecraft 数据集。

摘要:

神经视频游戏模拟器已成为生成和编辑视频的强大工具。其思想是将游戏表示为由智能体动作驱动的环境状态演化。虽然这种范式使用户能够逐动作玩游戏,但其刚性排除了更具语义形式的控制。为克服这一限制,我们用指定为一组自然语言动作和期望状态的提示来增强游戏模型。由此产生的 Promptable Game Model(PGM,可提示游戏模型)使用户能够通过用高层和低层动作序列提示它来玩游戏。最引人入胜的是,我们的 PGM 解锁了导演模式,在该模式中,通过以提示形式为智能体指定目标来玩游戏。这需要学习由我们的动画模型封装的“游戏 AI”,以利用高层约束在场景中导航、与对手对抗,并制定赢得一分的策略。为渲染得到的状态,我们使用由合成模型封装的组合式 NeRF 表示。为促进未来研究,我们提供了新收集、标注和校准的 Tennis 与 Minecraft 数据集。我们的方法在渲染质量方面显著优于现有神经视频游戏模拟器,并解锁了超出当前最先进技术能力的应用。我们的框架、数据和模型可在 https://snap-research.github.io/promptable-game-models/ 获取。

英文摘要:

Neural video game simulators emerged as powerful tools to generate and edit videos. Their idea is to represent games as the evolution of an environment's state driven by the actions of its agents. While such a paradigm enables users to play a game action-by-action, its rigidity precludes more semantic forms of control. To overcome this limitation, we augment game models with prompts specified as a set of natural language actions and desired states. The result-a Promptable Game Model (PGM)-makes it possible for a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. This requires learning "game AI", encapsulated by our animation model, to navigate the scene using high-level constraints, play against an adversary, and devise a strategy to win a point. To render the resulting state, we use a compositional NeRF representation encapsulated in our synthesis model. To foster future research, we present newly collected, annotated and calibrated Tennis and Minecraft datasets. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art. Our framework, data, and models are available at https://snap-research.github.io/promptable-game-models/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑