StereoWorld: 一种基于几何的单目到立体视频生成方法
StereoWorld: Geometry-Aware Monocular-to-Stereo Video Generation
AI总结:
StereoWorld通过几何感知正则化和时空拼接方案,实现高效高质量的单目到立体视频生成。
摘要:
随着XR设备的普及,对高质量立体视频的需求日益增加,但其生产仍然成本高昂且容易产生伪影。为了解决这一挑战,我们提出了StereoWorld,一种端到端框架,利用预训练的视频生成器进行高质量的单目到立体视频生成。我们的框架在单目视频输入的同时联合条件模型,并通过几何感知的正则化显式监督生成,以确保3D结构的准确性。此外,还集成了时空拼接方案,以实现高效、高分辨率的合成。为了支持大规模训练和评估,我们整理了一个高清立体视频数据集,包含超过1100万帧,对齐于自然人类瞳距(IPD)。大量实验表明,StereoWorld在视觉保真度和几何一致性方面显著优于先前方法。项目网页可在https://ke-xing.github.io/StereoWorld/上找到。
英文摘要:
The growing adoption of XR devices has fueled strong demand for high-quality stereo video, yet its production remains costly and artifact-prone. To address this challenge, we present StereoWorld, an end-to-end framework that repurposes a pretrained video generator for high-fidelity monocular-to-stereo video generation. Our framework jointly conditions the model on the monocular video input while explicitly supervising the generation with a geometry-aware regularization to ensure 3D structural fidelity. A spatio-temporal tiling scheme is further integrated to enable efficient, high-resolution synthesis. To enable large-scale training and evaluation, we curate a high-definition stereo video dataset containing over 11M frames aligned to natural human interpupillary distance (IPD). Extensive experiments demonstrate that StereoWorld substantially outperforms prior methods, generating stereo videos with superior visual fidelity and geometric consistency. The project webpage is available at https://ke-xing.github.io/StereoWorld/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。