SVG：通过去噪帧矩阵实现3D立体视频生成
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
AI总结：
本文提出一种无姿态、免训练的3D立体视频生成方法，通过深度变形和帧矩阵修复框架，并引入去遮挡边界重新注入方案，显著提升了立体视频的一致性和质量。
摘要：
视频生成模型在生成令人印象深刻的单目视频方面已展现出强大能力，然而3D立体视频的生成仍处于探索不足的状态。我们提出一种无姿态、免训练的方法，利用现成的单目视频生成模型来生成3D立体视频。我们的方法利用估计的视频深度，将生成的单目视频变形到立体基线下的相机视角，并采用一种新颖的帧矩阵视频修复框架。该框架利用视频生成模型来修复从不同时间戳和视角观察到的帧。这种有效的方法无需场景优化或模型微调，即可生成一致且语义连贯的立体视频。此外，我们开发了一种去遮挡边界重新注入方案，通过缓解潜在空间中去遮挡区域传播的负面影响，进一步提高了视频修复的质量。我们通过在来自各种生成模型（包括Sora [4]、Lumiere [2]、WALT [8]和Zeroscope [42]）的视频上进行实验，验证了所提方法的有效性。实验表明，我们的方法相比之前的方法有显著改进。代码将在\url{https://daipengwa.github.io/SVG_ProjectPage}发布。
英文摘要：
Video generation models have demonstrated great capabilities of producing impressive monocular videos, however, the generation of 3D stereoscopic video remains under-explored. We propose a pose-free and training-free approach for generating 3D stereoscopic videos using an off-the-shelf monocular video generation model. Our method warps a generated monocular video into camera views on stereoscopic baseline using estimated video depth, and employs a novel frame matrix video inpainting framework. The framework leverages the video generation model to inpaint frames observed from different timestamps and views. This effective approach generates consistent and semantically coherent stereoscopic videos without scene optimization or model fine-tuning. Moreover, we develop a disocclusion boundary re-injection scheme that further improves the quality of video inpainting by alleviating the negative effects propagated from disoccluded areas in the latent space. We validate the efficacy of our proposed method by conducting experiments on videos from various generative models, including Sora [4 ], Lumiere [2], WALT [8 ], and Zeroscope [ 42]. The experiments demonstrate that our method has a significant improvement over previous methods. The code will be released at \url{https://daipengwa.github.io/SVG_ProjectPage}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
