GaussVideoDreamer: 基于视频扩散和不一致感知高斯溅射的3D场景生成
GaussVideoDreamer: 3D Scene Generation with Video Diffusion and Inconsistency-Aware Gaussian Splatting
AI总结：
针对单图像3D场景重建的不适定性和现有方法的泛化或一致性问题，提出GaussVideoDreamer，通过渐进式视频补绘和不一致感知高斯溅射，实现更高质量、更快速度的3D场景生成，性能优于现有方法。
摘要：
单图像3D场景重建因其固有的不适定性和有限的输入约束而面临重大挑战。近期进展探索了两个有前景的方向：在3D一致数据集上训练但难以泛化到分布外场景的多视图生成模型，以及依赖深度数据或3D平滑度、存在跨视图不一致和次优错误处理问题的3D场景补绘与补全框架，这些问题最终会降低输出质量和计算性能。基于这些方法，我们提出GaussVideoDreamer，通过弥合图像、视频和3D生成之间的差距推进生成式多媒体方法，其核心创新包括：（1）利用时间连贯性提升多视图一致性和收敛速度的渐进式视频补绘策略；（2）用于以3D一致多视图证据引导视频扩散的3D高斯溅射一致性掩码。我们的流水线结合三个核心组件：几何感知初始化协议、不一致感知高斯溅射和渐进式视频补绘策略。实验结果表明，与现有方法相比，我们的方法实现了32%更高的LLaVA-IQA分数和至少2倍的速度提升，同时在多样场景中保持稳健性能。
英文摘要：
Single-image 3D scene reconstruction presents significant challenges due to its inherently ill-posed nature and limited input constraints. Recent advances have explored two promising directions: multiview generative models that train on 3D consistent datasets but struggle with out-of-distribution generalization, and 3D scene inpainting and completion frameworks that suffer from cross-view inconsistency and suboptimal error handling, as they depend exclusively on depth data or 3D smoothness, which ultimately degrades output quality and computational performance. Building upon these approaches, we present GaussVideoDreamer, which advances generative multimedia approaches by bridging the gap between image, video, and 3D generation, integrating their strengths through two key innovations: (1) A progressive video inpainting strategy that harnesses temporal coherence for improved multiview consistency and faster convergence. (2) A 3D Gaussian Splatting consistency mask to guide the video diffusion with 3D consistent multiview evidence. Our pipeline combines three core components: a geometry-aware initialization protocol, Inconsistency-Aware Gaussian Splatting, and a progressive video inpainting strategy. Experimental results demonstrate that our approach achieves 32% higher LLaVA-IQA scores and at least 2x speedup compared to existing methods while maintaining robust performance across diverse scenes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
