阿莱雅世界:交互式长视野世界建模——完整技术报告
AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report
- Alaya Lab(阿莱亚实验室)
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
研究旨在创建交互式视频世界模型,核心方法是基于15B视频扩散变压器构建阿莱雅世界,通过自回归生成短潜在块等实现多种能力,在iWorld - Bench上长视野生成性能最佳,为相关研究提供开源可扩展基础。
AI中文摘要:
与传统视频游戏开发不同,视频世界模型可根据用户输入即时生成交互式环境。实现这一愿景需要交互、持久时空一致性、稳定长视野生成和高效响应这四项紧密耦合的能力。我们展示了阿莱雅世界,这是一个交互式长视野视频世界模型,能以540p和720p生成24帧每秒的视频。它基于15B视频扩散变压器构建,在相机轨迹和可切换文本提示下自回归生成短潜在块。其有界视觉上下文结合了持久汇帧、压缩时间历史、几何对齐空间记忆和近期帧条件。为减少长期漂移,该模型用从自身展开中收集的损坏历史和预测残差进行训练。我们还引入了离散自回归蒸馏公式,将推理从约30个采样步骤减少到每个块4个步骤。在iWorld - Bench上,阿莱雅世界在长视野生成方面取得了最佳性能。阿莱雅世界被设想为一个全栈、开源和长期项目,旨在为交互式视频世界模型的未来研究提供可扩展基础。
英文摘要:
Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interactive environments from user inputs instantly. It enable us to create customized, explorable, and continuously evolving virtual world from text, an image, or video. Realizing this vision requires four tightly coupled capabilities: interaction, persistent spatiotemporal consistency, stable long-horizon generation, and efficient response. We present AlayaWorld, an interactive long-horizon video world model that generates 24-fps video at 540p and 720p. Built on a 15B video diffusion transformer, AlayaWorld generates short latent chunks autoregressively under camera trajectories and switchable text prompts. Its bounded visual context combines a persistent sink frame, compressed temporal history, geometry-aligned spatial memory, and recent-frame conditioning. To reduce long-term drift, the model is trained with corrupted histories and prediction residuals collected from its own roll-outs. We further introduce a discrete autoregressive distillation formulation that combines distribution-matching distillation, self-forcing++, and consistency distillation, reducing inference from approximately 30 sampling steps to four steps per chunk. On iWorld-Bench, AlayaWorld achieves the best performance over long-horizon generation. Conceived as a full-stack, open-source, and long-term project, AlayaWorld is intended to provide an extensible foundation for future research on interactive video world models.