发表机构
Alaya Lab; Peking University; Institute of Science Tokyo（Alaya 实验室; 北京大学; 东京科学大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SPW-Nav是一种流式全景世界模型，通过理解语言指令实时生成2K 360度视频，在相机跟随准确性和视频质量上优于现有方法，并支持即时指令切换。
AI 中文摘要
语言引导的全景视频生成有益于各种下游应用，如交互式3D场景探索、虚拟现实体验和具身智能体训练。现有的全景生成器遵循预定义轨迹，而交互式世界模型则通过透视视图中的低级动作进行运作。我们提出了SPW-Nav，一种流式全景世界模型，它能够理解移动指令，并从单个全景图实时流式生成一分钟的2K 360度视频。SPW-Nav将先前生成的全景图中的每条指令解释为相机运动。球面旋转解耦在球面上精确应用旋转，姿态对齐条件化使平移输入在长流中保持有界，而具有几步生成器的多术语记忆在指令变化时继续场景。我们还构建了SPW-NavSet，包含带有相机轨迹和验证指令的全景视频。在语言驱动下，SPW-Nav在相机跟随准确性和视频质量方面优于先前的全景生成器，并支持即时指令切换。
英文摘要
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.
CommentsProject page: https://alaya-lab.github.io/SPW-Nav
