PhyEdit: 通过物理基础图像编辑实现真实世界物体操作
PhyEdit: Towards Real-World Object Manipulation via Physically-Grounded Image Editing
AI总结：
PhyEdit通过结合显式几何模拟与2D-3D监督，提升图像编辑中物理准确性和操作一致性，引入RealManip-10K数据集和ManipEval基准测试，优于现有方法。
摘要：
在图像编辑中实现物理准确的物体操作对于交互式世界模型的应用至关重要。然而，现有视觉生成模型在精确空间操作上往往失败，导致物体缩放和定位错误。这一限制主要源于缺乏将3D几何和透视投影显式纳入机制。为此，我们开发了PhyEdit图像编辑框架，利用显式几何模拟作为上下文3D-aware视觉指导。通过结合这种即插即用的3D先验与联合2D-3D监督，我们的方法有效提升了物理准确性和操作一致性。为支持该方法并评估性能，我们提出了RealManip-10K真实世界数据集，用于3D-aware物体操作，包含配对图像和深度标注。我们还提出了ManipEval基准，包含多维指标以评估3D空间控制和几何一致性。大量实验表明，我们的方法在3D几何准确性和操作一致性上均优于现有方法，包括强大的闭源模型。
英文摘要：
Achieving physically accurate object manipulation in image editing is essential for its potential applications in interactive world models. However, existing visual generative models often fail at precise spatial manipulation, resulting in incorrect scaling and positioning of objects. This limitation primarily stems from the lack of explicit mechanisms to incorporate 3D geometry and perspective projection. To achieve accurate manipulation, we develop PhyEdit, an image editing framework that leverages explicit geometric simulation as contextual 3D-aware visual guidance. By combining this plug-and-play 3D prior with joint 2D--3D supervision, our method effectively improves physical accuracy and manipulation consistency. To support this method and evaluate performance, we present a real-world dataset, RealManip-40K, for 3D-aware object manipulation featuring paired images and depth annotations. We also propose ManipEval, a benchmark with multi-dimensional metrics to evaluate 3D spatial control and geometric consistency. Extensive experiments show that our approach outperforms existing methods, including strong closed-source models, in both 3D geometric accuracy and manipulation consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
