Thinking in Boxes: 真实图像中的3D编辑变得简单
Thinking in Boxes: 3D Editing in Real Images Made Easy
AI总结：
提出使用3D盒子作为结构化规范，通过用户提供输入和输出盒子来精确控制真实图像中的平移、旋转、缩放和视角变化，同时保持场景和物体身份，恢复未见的物体区域。
摘要：
文本和2D条件接口在图像编辑中提供对空间变换的弱、模糊控制——特别是在大物体运动和相机变化下。先前的工作使用了如盒子这样的3D基元，但仅作为松散的调节信号指示近似物体位置，而非指定变换。我们则使用3D盒子作为结构化规范：用户提供编辑的输入和输出盒子，将编辑视为一个适定的几何问题。这种“在盒子中思考”的界面，其中每个盒子面都带有颜色编码以传达3D方向，提供了对真实图像中平移、旋转、缩放和视角变化的精确控制，同时保留场景和物体身份，并恢复之前未见的物体区域。为了将变换与场景外观联系起来，我们引入了一个深度对齐的平面地板作为全局参考框架，并用深度感知线索进行着色。基于这种结构，图像生成器在大变换下产生一致的结果。该系统在两个阶段训练——在合成多物体场景和来自Objectron的小型真实世界视频集上——能够泛化到复杂的、野外真实图像。我们的方法直接作用于真实照片，并在大型3D编辑上显著优于最近的最先进方法。
英文摘要：
Text and 2D-conditioning interfaces provide weak, ambiguous control over spatial transformations in image editing -- particularly under large object motions and camera changes. Prior work has used 3D primitives such as boxes, but only as loose conditioning signals indicating approximate object location rather than specifying the transformation. We instead use 3D boxes as structured specifications: the user provides the input and output boxes of the edit, casting editing as a well-posed geometry problem. This ``thinking in boxes'' interface, where each box face is color-coded to convey 3D orientation, gives precise control over translation, rotation, scaling, and viewpoint changes in real images while preserving scene and object identity, and recovering previously unseen object regions. To ground transformations in scene appearance, we introduce a depth-aligned planar floor as a global reference frame, shaded with depth-aware cues. Conditioned on this structure, an image generator produces consistent results under large transformations. Trained in two stages -- on synthetic multi-object scenes and a small set of real-world videos from Objectron -- the system generalizes to complex, in-the-wild real images. Our method operates directly on real photographs and substantially outperforms recent state-of-the-art methods on large 3D edits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
