GeM-NR：面向非刚性场景变化的几何感知多视角编辑
GeM-NR: Geometry-Aware Multi-View Editing for Nonrigid Scene Changes
AI总结：
提出GeM-NR，一种无需训练的快速灵活方法，通过深度图对齐、视角投影和条件细化实现多视角一致的通用非刚性图像编辑，支持几何和外观的显著变化。
摘要：
近年来，基于生成模型的多视角图像编辑的发展使我们离通用3D内容生成和定制更近一步。现有大多数工作通过利用未编辑场景的几何结构，专注于刚性或仅外观的编辑。这自然将这些方法限制在保留底层场景结构的编辑上。其他方法则针对特定图像编辑任务（如物体移除和添加）进行训练。尽管取得了进展，但通用的非刚性编辑（即大幅改变场景几何的编辑）对现有方法仍然具有挑战性。我们提出GeM-NR，一种快速灵活且无需训练的方法，用于通用的多视角一致图像编辑，包括大幅改变场景几何和外观的编辑。给定一个使用选定骨干编辑器（如FLUX、Qwen、BrushNet）编辑的锚点图像和一个未编辑的查询图像，GeM-NR以与锚点编辑一致的方式编辑查询图像。该方法包含多个阶段：(i) 深度图估计，我们提出一种策略以最大化编辑和未编辑场景的3D点云之间的对齐；(ii) 投影到查询视角；(iii) 基于未编辑查询的条件细化所得图像。基于条件化的公式从两个视角很好地扩展到物体的多个视角。我们展示了该方法处理几何和外观显著变化的编辑的能力，这是现有方法难以做到的。我们进行了广泛评估，表明我们的方法在各种编辑任务中提高了一致性，包括生成编辑场景的3D表示。定量和定性结果均表明，我们的方法在编辑质量以及多视角几何和光度一致性方面达到了最先进的性能。
英文摘要：
Recent developments in multi-view image editing with generative models have brought us a step closer toward general 3D content generation and customization. Most existing works focus on rigid or appearance-only edits by utilizing the geometry of the unedited scene. This naturally limits these methods to edits that preserve the underlying scene structure. Current nonrigid approaches are limited to object removal and insertion, reflecting the data they are trained on. General nonrigid edits, i.e., edits that substantially and arbitrarily change the scene geometry, remain challenging for existing methods. We propose GeM-NR, a fast and flexible training-free approach for general multi-view consistent image editing, including edits that drastically change the geometry and appearance of the scene. Given an anchor image edited with a chosen 2D editor and a query unedited image, GeM-NR edits the query image consistently with the anchor edit. The method incorporates multiple stages: (i) depth map estimation, where we propose a strategy to maximize the alignment between the 3D point clouds of the edited and unedited scenes, (ii) projection onto a query viewpoint, and (iii) refinement of the obtained image conditioned on the unedited query. We demonstrate the ability of our method to handle edits with significant changes in geometry and appearance, something that existing methods struggle with. We perform an extensive evaluation showing that GeM-NR improves consistency for a wide variety of edit tasks, including generating 3D representations of the edited scene. Both quantitative and qualitative results indicate the state-of-the-art performance of our method in terms of edit quality as well as geometric and photometric consistency across multiple views.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
