arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2312.08563cs.CV

高效NeRF2NeRF:利用多视图对应增强扩散模型简化文本驱动的3D编辑

Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Models

Apple(苹果公司);University at Buffalo(布法罗大学);UT Austin(得克萨斯大学奥斯汀分校)

Liangchen Song, Liangliang Cao, Jiatao Gu, Yifan Jiang, Junsong Yuan, Hao Tang

AI总结:

提出将对应正则化引入扩散模型,利用多视图一致性加速文本驱动的3D编辑,实现10倍提速和2分钟内完成编辑。

摘要:

文本驱动的3D内容编辑的进步得益于2D生成扩散模型的发展。然而,阻碍3D内容编辑广泛采用的一个主要障碍是其耗时的处理过程。这一挑战源于从基于2D图像的生成模型获得一致的3D输出所需的迭代和细化步骤。近期最先进的方法通常需要几分钟到几小时的优化时间,才能使用单个GPU编辑一个3D场景。在这项工作中,我们提出通过将对应正则化引入扩散模型,可以显著加速3D编辑过程。这一方法受到以下概念的启发:扩散过程中的估计样本在扩散生成过程中应保持多视图一致性。通过利用这种多视图一致性,我们可以以更快的速度编辑3D内容。在大多数场景中,我们提出的技术相比基线方法带来了10倍的加速,并在2分钟内完成3D场景的编辑,且质量相当。

英文摘要:

The advancement of text-driven 3D content editing has been blessed by the progress from 2D generative diffusion models. However, a major obstacle hindering the widespread adoption of 3D content editing is its time-intensive processing. This challenge arises from the iterative and refining steps required to achieve consistent 3D outputs from 2D image-based generative models. Recent state-of-the-art methods typically require optimization time ranging from tens of minutes to several hours to edit a 3D scene using a single GPU. In this work, we propose that by incorporating correspondence regularization into diffusion models, the process of 3D editing can be significantly accelerated. This approach is inspired by the notion that the estimated samples during diffusion should be multiview-consistent during the diffusion generation process. By leveraging this multiview consistency, we can edit 3D content at a much faster speed. In most scenarios, our proposed technique brings a 10$\times$ speed-up compared to the baseline method and completes the editing of a 3D scene in 2 minutes with comparable quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑