3D-GOI：用于多方面和多对象编辑的3D GAN全反转
3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing
AI总结：
针对现有GAN inversion方法忽略空间信息且仅限单对象编辑的局限，提出3D-GOI框架，通过反转GIRAFFE控制的属性码并采用分割、粗略编码提取与轮转优化算法，首次实现了多对象的多方面仿射信息编辑。
摘要：
当前的GAN inversion方法通常只能编辑单个对象和背景的外观与形状，而忽略了空间信息。在本研究中，我们提出了一个3D编辑框架3D-GOI，以实现对多个对象的仿射信息（缩放、平移和旋转）的多方面编辑。3D-GOI通过反转由著名的3D GAN模型GIRAFFE控制的大量属性码（对象形状/外观/缩放/旋转/平移，背景形状/外观以及相机姿态）来实现复杂的编辑功能。准确反转所有码具有挑战性，3D-GOI遵循三个主要步骤来解决这一挑战。首先，我们在多对象图像中分割对象和背景。其次，我们使用定制的Neural Inversion Encoder获取每个对象的粗略码。最后，我们使用轮转优化算法获取精确的码以重建图像。据我们所知，3D-GOI是首个实现多对象多方面编辑的框架。定性和定量实验均表明，3D-GOI在复杂多对象场景中进行灵活、多方面编辑方面具有巨大潜力。我们的项目和代码已在https://3d-goi.github.io发布。
英文摘要：
The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI, to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/appearance/scale/rotation/translation, background shape/appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes.Our project and code are released at https://3d-goi.github.io .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
