发表机构
School of Electronic and Computer Engineering, Peking University; Pengcheng Laboratory; Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology（北京大学电子与计算机工程学院; 鹏城实验室; 广东省超高清沉浸式媒体技术重点实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对输入视图有限时3DGS重建质量差的问题，提出DualDiff3D框架，采用双扩散先验与SAA模块，结合RRO循环提升性能，优于现有最优方法。
AI 中文摘要
尽管3D Gaussian Splatting（3DGS）已彻底变革了3D重建与新视图合成，但输入视图有限的场景常会导致重建质量不佳，渲染的新视图出现伪影。近期研究尝试利用强大的扩散先验，但通常在单一网络中沿额外维度拼接渲染视图与参考视图进行处理，忽视了不同视图应保持外观相似性但因视角偏移存在结构差异的固有特性，进而因两种特性的冲突导致模糊。本文提出DualDiff，一种利用双扩散先验与结构-外观注意力（SAA）模块的新型流水线，引入参考引导以优化从有缺陷3D表示渲染的低质量新视图。具体而言，保留一个扩散分支专注于从低质量新视图提取结构信息，引入另一分支确保与参考视图的外观一致性。此外，本文提出名为DualDiff3D的3D重建框架，集成可靠性增强的渲染-优化-优化（RRO）循环，逐步且鲁棒地融入优化后的新视图，生成更准确的3DGS。大量实验表明，本文方法即便在仅推理设置下也优于现有最优方法，通过训练可进一步提升性能。代码与预训练权重可在this https URL获取。
英文摘要
While 3D Gaussian Splatting (3DGS) has revolutionized 3D reconstruction and novel-view synthesis, scenarios with limited input views often lead to poor reconstruction quality and artifacts in rendered novel views. Recent efforts attempt to utilize powerful diffusion priors, yet they typically process rendered and reference views concatenated along an additional dimension in a single network. These methods overlook an inherent nature that different views should maintain appearance similarity but differ in structure due to view shifts, leading to blur caused by conflicts between the two properties. In this paper, we propose DualDiff, a novel pipeline that leverages dual diffusion priors with a Structure-Appearance Attention (SAA) module to introduce reference guidance for refining low-quality novel views rendered from flawed 3D representations. Specifically, we retain one diffusion branch to focus on extracting structural information from the low-quality novel views, while introducing another branch to ensure appearance consistency with reference views. Furthermore, we present a 3D reconstruction framework named DualDiff3D, which integrates a reliability-enhanced Render-Refine-Optimize (RRO) loop to progressively and robustly incorporate the refined novel views, yielding more accurate 3DGS. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods even in the inference-only setting, with further performance gains achievable through training. Our code and pre-trained weights are available at https://github.com/Akaneqwq/DualDiff3D.
