局部3D编辑：基于CLIP知识的3D蒸馏
Local 3D Editing via 3D Distillation of CLIP Knowledge
AI总结：
提出LENeRF，通过CLIP知识蒸馏到3D空间实现仅需文本输入的局部3D编辑，利用三个附加模块估计3D注意力场，实现细粒度局部操作。
摘要：
3D内容操作是计算机视觉中的一项重要任务，具有许多实际应用（例如，产品设计、卡通生成和3D头像编辑）。最近提出的3D GANs可以利用神经辐射场（NeRF）生成多样化的逼真3D内容。然而，NeRF的操作仍然是一个具有挑战性的问题，因为操作后视觉质量往往会下降，并且使用诸如2D语义图之类的次优控制手柄进行操作。虽然文本引导的操作在3D编辑中显示出潜力，但此类方法通常缺乏局部性。为了克服这些问题，我们提出了局部编辑NeRF（LENeRF），它仅需要文本输入即可进行细粒度和局部化的操作。具体来说，我们提出了LENeRF的三个附加模块：潜在残差映射器、注意力场网络和变形网络，它们通过估计3D注意力场来联合用于3D特征的局部操作。3D注意力场以无监督方式学习，通过将CLIP的零样本掩码生成能力蒸馏到3D空间，并辅以多视图引导。我们进行了多样化的实验，并在定量和定性方面进行了全面评估。
英文摘要：
3D content manipulation is an important computer vision task with many real-world applications (e.g., product design, cartoon generation, and 3D Avatar editing). Recently proposed 3D GANs can generate diverse photorealistic 3D-aware contents using Neural Radiance fields (NeRF). However, manipulation of NeRF still remains a challenging problem since the visual quality tends to degrade after manipulation and suboptimal control handles such as 2D semantic maps are used for manipulations. While text-guided manipulations have shown potential in 3D editing, such approaches often lack locality. To overcome these problems, we propose Local Editing NeRF (LENeRF), which only requires text inputs for fine-grained and localized manipulation. Specifically, we present three add-on modules of LENeRF, the Latent Residual Mapper, the Attention Field Network, and the Deformation Network, which are jointly used for local manipulations of 3D features by estimating a 3D attention field. The 3D attention field is learned in an unsupervised way, by distilling the zero-shot mask generation capability of CLIP to the 3D space with multi-view guidance. We conduct diverse experiments and thorough evaluations both quantitatively and qualitatively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
