MUGEN：通过相机控制进行交互式全景世界探索
MUGEN: Interactive Panoramic World Exploration via Camera Control
浏览论文内容
中文总结 AI 辅助
本文提出MUGEN数据集和Wan360模型，解决交互式全景视频生成中数据与模型差距，实现高质量、相机可控的360度视频生成。
中文摘要 AI 辅助
交互式全景视频生成旨在合成沉浸式360°视频，这些视频在探索过程中遵循用户指定的相机轨迹时保持视觉连贯性。然而，该领域的进展受到数据与模型双重差距的限制：现有的全景视频数据集通常较短、标注较弱或缺乏相机轨迹，而现有的相机控制视频生成模型是为透视视频设计的，并不直接支持全景几何。在本文中，我们引入MUGEN和Wan360来解决这些局限性。MUGEN是一个大规模真实世界全景视频数据集，专为交互式360度世界探索而设计，包含超过1300小时至少4K分辨率的全景视频，并带有丰富的语义和几何标注。基于MUGEN，我们进一步提出了Wan360，一个相机可控的交互式全景视频生成模型。全景视频通常通过等距柱状投影（ERP）表示，该投影将球面360度视图展开为具有周期性经度接缝和极点失真的矩形帧。为此，Wan360引入了三个无参数的ERP感知组件：周期性经度RoPE用于接缝一致的位置编码，ERP感知填充用于减少边界伪影，以及随机滚动偏航用于一致性学习。对于相机控制，Wan360使用全景Plücker嵌入，通过ERP射线而非透视针孔射线来表示相机运动。实验表明，MUGEN作为全景世界探索的数据基础，而Wan360能够实现高质量、时间连贯、相机可控的360度视频生成。
英文摘要
Interactive panoramic video generation aims to synthesize immersive 360\textdegree{} videos that remain visually coherent while following user-specified camera trajectories during exploration. However, progress is limited by a coupled data-and-model gap: existing panoramic video datasets are often short, weakly annotated, or lack camera trajectories, while existing camera-controlled video generation models are designed for perspective videos and do not directly support panoramic geometry. In this paper, we introduce MUGEN and Wan360 to address these limitations. MUGEN is a large-scale real-world panoramic video dataset tailored to interactive 360-degree world exploration, comprising over 1,300 hours of at least 4K panoramic videos with rich semantic and geometric annotations. Built on MUGEN, we further present Wan360, a camera-controllable interactive panoramic video generation model. Panoramic videos are commonly represented by EquiRectangular Projection (ERP), which unfolds a spherical 360-degree view into a rectangular frame with cyclic longitude seams and pole distortions. To this end, Wan360 introduces three parameter-free ERP-aware components: periodic longitude RoPE for seam-consistent positional encoding, ERP-aware padding for reducing boundary artifacts, and random roll yaw for consistent learning. For camera control, Wan360 uses a panoramic Plücker embedding that represents camera motion with ERP rays rather than perspective pinhole rays. Experiments show that MUGEN serves as a data foundation for panoramic world exploration, and that Wan360 enables high-quality, temporally coherent, camera-controllable 360-degree video generation.
发表机构
- Alaya Lab（Alaya 实验室）
- Beijing Institute of Technology（北京理工大学）
- Shanghai Innovation Institute（上海创新研究院）
机构由 AI 辅助整理，请以论文原文为准。
