Perturb-and-Revise:基于生成轨迹的灵活3D编辑
Perturb-and-Revise: Flexible 3D Editing with Generative Trajectories
AI总结:
针对现有3D编辑方法难以处理大幅几何与外观变化的问题,提出Perturb-and-Revise方法,通过局部损失景观自适应扰动NeRF参数、结合生成轨迹与身份保持梯度修正,实现灵活一致的多维度3D编辑。
摘要:
基于文本的扩散模型的最新进展推动了3D重建和文本驱动3D编辑的进步。现有3D编辑方法虽擅长修改颜色、纹理和风格,但难以应对大幅几何或外观变化,限制了其应用场景。为此,我们提出Perturb-and-Revise方法,可实现多种NeRF(神经辐射场)编辑。首先,我们用随机初始化扰动NeRF参数以生成通用初始化,扰动程度通过分析局部损失景观自动确定;随后通过生成轨迹修正编辑后的NeRF,并在生成过程中加入身份保持梯度以优化结果。大量实验表明,Perturb-and-Revise可灵活、有效且一致地编辑3D对象的颜色、外观和几何形状。360°效果请访问项目页面:https://susunghong.github.io/Perturb-and-Revise。
英文摘要:
Recent advancements in text-based diffusion models have accelerated progress in 3D reconstruction and text-based 3D editing. Although existing 3D editing methods excel at modifying color, texture, and style, they struggle with extensive geometric or appearance changes, thus limiting their applications. To this end, we propose Perturb-and-Revise, which makes possible a variety of NeRF editing. First, we perturb the NeRF parameters with random initializations to create a versatile initialization. The level of perturbation is determined automatically through analysis of the local loss landscape. Then, we revise the edited NeRF via generative trajectories. Combined with the generative process, we impose identity-preserving gradients to refine the edited NeRF. Extensive experiments demonstrate that Perturb-and-Revise facilitates flexible, effective, and consistent editing of color, appearance, and geometry in 3D. For 360° results, please visit our project page: https://susunghong.github.io/Perturb-and-Revise.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。