迈向免训练的三维场景编辑方法
Towards a Training Free Approach for 3D Scene Editing
AI总结：
针对现有NeRF三维编辑需逐次训练且无法实时的问题，提出免训练的FreeEdit方法，以网格表示替代NeRF，利用基础模型实现插入、替换和删除操作，并引入算法确定最佳放置位置。
摘要：
文本驱动的扩散模型在图像编辑方面展现出了卓越的能力。然而，在编辑三维场景时，现有工作大多依赖训练神经辐射场（NeRF）进行三维编辑。最近的NeRF编辑方法通过部署二维扩散模型来利用编辑操作，并将这些编辑投影到三维空间中。它们需要强位置先验以及文本提示来识别编辑位置。这些方法仅适用于小型三维场景，并且更针对特定场景。它们需要针对每次特定编辑进行训练，无法用于实时编辑。为解决这些局限，我们提出了一种新方法FreeEdit，采用网格表示替代NeRF，以免训练的方式进行编辑。由于基础模型空间取得的进展，免训练方法如今成为可能。我们利用这些模型提供一种免训练的替代方案，并针对插入、替换和删除提出了解决方案。我们将插入、替换和删除视为基本操作块，通过这些操作的特定组合可执行复杂编辑。给定文本提示和三维场景，我们的模型能够识别应插入、替换或删除的对象，以及应执行编辑的位置。作为FreeEdit的一部分，我们还引入了一种新算法，用于在定位对象上找到最佳放置位置。我们通过定量和定性指标，在广泛场景上与基线模型进行比较来评估我们的模型，并展示了我们方法相对于其他方法的优势。
英文摘要：
Text driven diffusion models have shown remarkable capabilities in editing images. However, when editing 3D scenes, existing works mostly rely on training a NeRF for 3D editing. Recent NeRF editing methods leverages edit operations by deploying 2D diffusion models and project these edits into 3D space. They require strong positional priors alongside text prompt to identify the edit location. These methods are operational on small 3D scenes and are more generalized to particular scene. They require training for each specific edit and cannot be exploited in real-time edits. To address these limitations, we propose a novel method, FreeEdit, to make edits in training free manner using mesh representations as a substitute for NeRF. Training-free methods are now a possibility because of the advances in foundation model's space. We leverage these models to bring a training-free alternative and introduce solutions for insertion, replacement and deletion. We consider insertion, replacement and deletion as basic blocks for performing intricate edits with certain combinations of these operations. Given a text prompt and a 3D scene, our model is capable of identifying what object should be inserted/replaced or deleted and location where edit should be performed. We also introduce a novel algorithm as part of FreeEdit to find the optimal location on grounding object for placement. We evaluate our model by comparing it with baseline models on a wide range of scenes using quantitative and qualitative metrics and showcase the merits of our method with respect to others.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
