从像素到状态：将交互式世界模型重新思考为游戏引擎

中文摘要 AI 辅助

构建能连贯响应玩家动作的交互式世界一直是计算机图形学、游戏和人工智能的共同目标。近期视频生成模型通过预测基于用户动作的未来观察为实现此目标提供了数据驱动路径，且渐被视为潜在的下一代游戏引擎。然而，要实现真正交互式游戏世界，需满足诸多条件。本文以循环动作 - 状态 - 观察循环为视角，从玩家动作控制、游戏状态动态、状态 - 观察持久性和实时交互式生成这四个维度审视交互式游戏世界建模，分析各维度现有方法的优缺点。此外，还为《黑神话：悟空》提供了一个可扩展数据引擎，收集了超90小时带有帧对齐玩家动作、真实游戏状态、视觉观察及结构化和语义注释的游戏玩法，作为状态感知游戏世界建模的资源，希望推动交互式游戏世界发展。

英文摘要

Building interactive worlds that respond coherently to player actions has long been a shared goal of computer graphics, games, and artificial intelligence. Recent video generative models provide a data-driven route toward this goal by predicting future observations conditioned on user actions, and are increasingly regarded as potential next-generation game engines. Realizing a genuinely interactive game world, however, requires interaction outcomes that follow rules over evolving game conditions, consequences that persist over long horizons, and a generation loop that operates in real time. Conventional game engines realize these properties through a recurrent action-state-observation loop, in which player actions update an explicit game state according to predefined rules and observations are rendered from the resulting state. Taking this loop as an organizing lens, this paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation. For each dimension, we start from the capabilities required by an interactive game world, group existing approaches into representative families, and discuss the strengths and trade-offs of each family. Complementing this analysis, we present a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay with frame-aligned player actions, ground-truth game states, and visual observations, together with structured and semantic annotations, as a resource for state-aware game world modeling. We hope this paper offers a clear picture of where the field stands and fosters progress toward interactive game worlds.

发表机构

• Alaya Lab（阿亚实验室）

机构由 AI 辅助整理，请以论文原文为准。