通过文本提示无需重新训练即可编辑三维场景
Editing 3D Scenes via Text Prompts without Retraining
AI总结：
本文提出DN2N文本驱动三维场景编辑方法，通过利用二维图像编辑模型、过滤不一致图像并引入跨视图正则化，使NeRF模型无需重新训练即可实现多种编辑效果并泛化到新场景。
摘要：
近年来，大量扩散模型已被应用于图像合成与编辑。然而，三维场景编辑仍处于早期阶段，面临诸多挑战，例如需要为不同类型的编辑设计特定方法、为各种三维场景重新训练新模型，以及编辑过程中缺乏便捷的人机交互。为解决这些问题，我们提出了一种文本驱动的编辑方法，称为DN2N，它允许直接获得具有通用编辑能力的NeRF模型，从而无需重新训练。我们的方法采用现成的基于文本的二维图像编辑模型来修改三维场景图像，随后通过一个过滤过程丢弃那些破坏三维一致性的、编辑质量较差的图像。然后，我们将剩余的编辑不一致性视为去除噪声扰动的问题，该问题可以通过生成具有相似扰动特征的训练数据进行训练来解决。我们进一步提出了跨视图正则化项，以帮助泛化的NeRF模型减轻这些扰动。我们的文本驱动方法允许用户根据自己期望的描述来编辑三维场景，与先前工作相比更加友好、直观和实用。实验结果表明，我们的方法实现了多种编辑类型，包括但不限于外观编辑、天气转换、材质更改和风格迁移。最重要的是，我们的方法具有良好的泛化能力，其编辑能力由一组模型参数共享，无需为某些特定场景定制编辑模型，从而能够直接从用户输入推断出具有编辑效果的新视图。项目网站可在https://sk-fun.fun/DN2N访问。
英文摘要：
Numerous diffusion models have recently been applied to image synthesis and editing. However, editing 3D scenes is still in its early stages. It poses various challenges, such as the requirement to design specific methods for different editing types, retraining new models for various 3D scenes, and the absence of convenient human interaction during editing. To tackle these issues, we introduce a text-driven editing method, termed DN2N, which allows for the direct acquisition of a NeRF model with universal editing capabilities, eliminating the requirement for retraining. Our method employs off-the-shelf text-based editing models of 2D images to modify the 3D scene images, followed by a filtering process to discard poorly edited images that disrupt 3D consistency. We then consider the remaining inconsistency as a problem of removing noise perturbation, which can be solved by generating training data with similar perturbation characteristics for training. We further propose cross-view regularization terms to help the generalized NeRF model mitigate these perturbations. Our text-driven method allows users to edit a 3D scene with their desired description, which is more friendly, intuitive, and practical than prior works. Empirical results show that our method achieves multiple editing types, including but not limited to appearance editing, weather transition, material changing, and style transfer. Most importantly, our method generalizes well with editing abilities shared among a set of model parameters without requiring a customized editing model for some specific scenes, thus inferring novel views with editing effects directly from user input. The project website is available at https://sk-fun.fun/DN2N
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
