OBJECT 3DIT: Language-guided 3D-aware Image Editing（语言引导的3D感知图像编辑）
OBJECT 3DIT: Language-guided 3D-aware Image Editing
AI总结：
针对现有图像编辑工具忽略3D几何结构的问题，本研究提出语言引导的3D感知编辑任务，发布含40万样本的OBJECT数据集，并提出3DIT模型，仅用合成数据训练即可泛化到真实图像编辑。
摘要：
现有图像编辑工具虽然功能强大，通常却忽略了图像投射所依据的底层3D几何结构。因此，使用这些工具进行的编辑可能会脱离作为图像形成过程基础的几何结构与光照条件。在本研究中，我们提出了语言引导的3D感知编辑这一新任务，要求依据底层3D场景的语境，按照语言指令编辑图像中的物体。为推动该方向的研究进展，我们发布了OBJECT数据集：该数据集包含40万个由程序化生成的3D场景创建的编辑样本，每个样本包含一张输入图像、一条语言编辑指令和一张编辑后的图像。我们还提出了3DIT：面向四项编辑任务的单任务与多任务模型。我们的模型展现出出色的能力，能够理解整个场景的3D构成，将周围物体、表面、光照条件、阴影以及物理上合理的物体配置纳入考量。出人意料的是，仅使用OBJECT数据集中的合成场景训练，3DIT的编辑能力就能泛化到真实世界图像上。
英文摘要：
Existing image editing tools, while powerful, typically disregard the underlying 3D geometry from which the image is projected. As a result, edits made using these tools may become detached from the geometry and lighting conditions that are at the foundation of the image formation process. In this work, we formulate the newt ask of language-guided 3D-aware editing, where objects in an image should be edited according to a language instruction in context of the underlying 3D scene. To promote progress towards this goal, we release OBJECT: a dataset consisting of 400K editing examples created from procedurally generated 3D scenes. Each example consists of an input image, editing instruction in language, and the edited image. We also introduce 3DIT : single and multi-task models for four editing tasks. Our models show impressive abilities to understand the 3D composition of entire scenes, factoring in surrounding objects, surfaces, lighting conditions, shadows, and physically-plausible object configurations. Surprisingly, training on only synthetic scenes from OBJECT, editing capabilities of 3DIT generalize to real-world images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
