DisCo3D:用于3D场景编辑的多视图一致性蒸馏
DisCo3D: Distilling Multi-View Consistency for 3D Scene Editing
AI总结:
本文提出DisCo3D框架,通过微调3D生成器、一致性蒸馏训练2D编辑器并结合Gaussian Splatting,实现了稳定的3D场景编辑多视图一致性,且编辑质量优于现有SOTA方法。
摘要:
尽管扩散模型在2D图像生成与编辑领域已取得显著进展,但将此类能力扩展至3D编辑仍具挑战,尤其难以维持多视图一致性。传统方法通常基于单个编辑视图通过迭代细化更新3D表示,然而这类方法常因跨视图不一致性出现收敛缓慢、生成模糊伪影的问题。近期方法通过传播2D编辑注意力特征提升效率,但受限于约束不足,在复杂场景中仍存在细粒度不一致性与失效模式。为解决该问题,本文提出一种名为DisCo3D的新型框架,将3D一致性先验蒸馏至2D编辑器中。该方法首先使用多视图输入微调3D生成器以完成场景适配,随后通过一致性蒸馏训练2D编辑器,最终将编辑后的多视图输出通过Gaussian Splatting优化为3D表示。实验结果表明,DisCo3D可实现稳定的多视图一致性,且在编辑质量上优于现有最先进方法。
英文摘要:
While diffusion models have demonstrated remarkable progress in 2D image generation and editing, extending these capabilities to 3D editing remains challenging, particularly in maintaining multi-view consistency. Classical approaches typically update 3D representations through iterative refinement based on a single editing view. However, these methods often suffer from slow convergence and blurry artifacts caused by cross-view inconsistencies. Recent methods improve efficiency by propagating 2D editing attention features, yet still exhibit fine-grained inconsistencies and failure modes in complex scenes due to insufficient constraints. To address this, we propose \textbf{DisCo3D}, a novel framework that distills 3D consistency priors into a 2D editor. Our method first fine-tunes a 3D generator using multi-view inputs for scene adaptation, then trains a 2D editor through consistency distillation. The edited multi-view outputs are finally optimized into 3D representations via Gaussian Splatting. Experimental results show DisCo3D achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。