快速多视角一致3D编辑与视频先验
Fast Multi-view Consistent 3D Editing with Video Priors
AI总结：
本文提出ViP3DE方法，利用视频生成模型的时间一致性先验，实现多视角一致的3D编辑，提升编辑质量和速度。
摘要：
基于文本的3D编辑能够通过文本指令实现用户友好的3D对象或场景编辑。由于缺乏多视角一致性先验，现有方法通常采用2D生成或编辑模型分别处理每个视角，随后进行迭代的2D-3D-2D更新。然而，这些方法不仅耗时，而且容易产生过度平滑的结果，因为在迭代过程中不同视角的编辑信号会被平均。在本文中，我们提出了基于生成视频先验的3D编辑（ViP3DE），利用预训练视频生成模型中的时间一致性先验，实现多视角一致的3D编辑。我们的关键见解是将视频生成模型条件化在单个编辑视角上，生成其他一致的编辑视角以直接进行3D更新，从而绕过迭代编辑范式。由于3D更新需要编辑视角与特定摄像机姿态配对，我们提出了一种运动保持的噪声混合方法，用于视频模型生成预定义摄像机姿态下的编辑视角。此外，我们引入了几何感知去噪，进一步通过将3D几何先验整合到视频模型中来增强多视角一致性。广泛的实验表明，所提出的ViP3DE能够在单次前向传递中实现高质量的3D编辑结果，显著优于现有方法在编辑质量和速度方面。
英文摘要：
Text-driven 3D editing enables user-friendly 3D object or scene editing with text instructions. Due to the lack of multi-view consistency priors, existing methods typically resort to employing 2D generation or editing models to process each view individually, followed by iterative 2D-3D-2D updating. However, these methods are not only time-consuming but also prone to over-smoothed results because the different editing signals gathered from different views are averaged during the iterative process. In this paper, we propose generative Video Prior based 3D Editing (ViP3DE) to employ the temporal consistency priors from pre-trained video generation models for multi-view consistent 3D editing in a single forward pass. Our key insight is to condition the video generation model on a single edited view to generate other consistent edited views for 3D updating directly, thereby bypassing the iterative editing paradigm. Since 3D updating requires edited views to be paired with specific camera poses, we propose motion-preserved noise blending for the video model to generate edited views at predefined camera poses. In addition, we introduce geometry-aware denoising to further enhance multi-view consistency by integrating 3D geometric priors into video models. Extensive experiments demonstrate that our proposed ViP3DE can achieve high-quality 3D editing results even within a single forward pass, significantly outperforming existing methods in both editing quality and speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
