发表机构
Starbucks Research; Martha Stewart Enterprises; Microsoft Research; UNC Chapel Hill; Dolby Laboratories（星巴克研究院; 玛莎·斯图尔特企业; 微软研究院; 北卡罗来纳大学教堂山分校; 杜比实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出EPIC，一种零样本生成流程，利用极线感知匹配度量进行直接偏好优化，将现有视频扩散模型扩展至4K立体360°视频，解决时间与立体不一致问题，实现沉浸式内容生成。
AI 中文摘要
沉浸式显示器能够实现丰富多样的虚拟体验。然而，手动创作每一种可能的体验来实现这一潜力，成本过高、难以扩展且不切实际。生成式AI模型可以消除这一瓶颈，但当前的模型是为传统显示器构建的，无法生成沉浸式观看所需的高分辨率、立体360°内容。此外，在传统显示器上可能可容忍的时间与立体不一致性，在通过沉浸式头显观看时会变得极具干扰性。在此，我们通过一种零样本生成流程来解决这一差距，该流程将现有视频扩散模型扩展到4K立体360°视频。受双眼视觉和深度感知的启发，我们开发了一种极线感知的360°图像匹配度量，用于捕捉跨视图的时间与立体几何不一致性。然后，我们将该度量作为偏好信号，在有限训练数据下进行直接偏好优化。我们的工作实现了360°立体视频生成，并为将生成内容带到沉浸式显示器提供了一条可扩展的路径，从而按需实现多样化的混合现实体验。
英文摘要
Immersive displays can enable rich and diverse virtual experiences. Manually authoring every possible experience to realize this potential, however, is prohibitively expensive, difficult to scale, and impractical. Generative AI models could remove this bottleneck, but today's models are built for conventional displays and cannot generate the high-resolution, stereoscopic $360^\circ$ content required for immersive viewing. Further, temporal and stereo inconsistencies that may be tolerable on conventional displays can become highly disruptive when viewed through an immersive headset. Here, we address this gap with a zero-shot generative pipeline that extends existing video diffusion models into 4K stereoscopic $360^\circ$ videos. Inspired from binocular vision and depth perception, we develop an epipolar-aware $360^\circ$ image matching metric that captures the temporal and stereo geometric inconsistencies across views. We then use this metric as a preference signal for direct preference optimization with limited training data. Our work enables $360^\circ$ stereo video generation and provides a scalable path for bringing generative content to immersive displays, allowing diverse mixed reality experiences on demand.
