RotationDrag：基于旋转扩散特征的点式图像编辑
RotationDrag: Point-based Image Editing with Rotated Diffusion Features
AI总结：
本文发现扩散模型特征在平面内旋转下变化剧烈，提出RotationDrag方法利用旋转图像特征图精确跟踪控制点以提升点式图像编辑性能，并构建首个平面内旋转基准RotateBench验证其优越性。
摘要：
在保持图像保真度的同时精确且用户友好地操控图像内容，一直是图像编辑领域的关键问题。得益于生成模型的强大能力，近期的点式图像编辑方法允许用户通过点击若干控制点，以高泛化能力交互式地改变图像内容。但上述编辑过程通常基于一个假设：在从初始点到目标点的运动监督步骤中，特征保持不变。本文对扩散模型的特征空间进行了全面研究，发现特征在平面内旋转下会发生剧烈变化。基于此，我们提出了一种名为RotationDrag的新方法，当用户希望对图像内容进行平面内旋转时，该方法能显著提升点式图像编辑性能。我们的方法通过利用旋转图像的特征图更精确地跟踪控制点，从而确保精确优化和高图像保真度。此外，我们构建了一个专注于平面内旋转的基准RotateBench，这是首个在真实图像和生成图像上评估点式图像编辑方法在平面内旋转场景下表现的基准。一项全面的用户研究表明，与DragDiffusion基线及其他现有基于扩散的方法相比，本方法在完成用户期望的平面内旋转方面具有卓越能力。代码和实验结果见项目页面 https://github.com/Tony-Lowe/RotationDrag。
英文摘要：
A precise and user-friendly manipulation of image content while preserving image fidelity has always been crucial to the field of image editing. Thanks to the power of generative models, recent point-based image editing methods allow users to interactively change the image content with high generalizability by clicking several control points. But the above mentioned editing process is usually based on the assumption that features stay constant in the motion supervision step from initial to target points. In this work, we conduct a comprehensive investigation in the feature space of diffusion models, and find that features change acutely under in-plane rotation. Based on this, we propose a novel approach named RotationDrag, which significantly improves point-based image editing performance when users intend to in-plane rotate the image content. Our method tracks handle points more precisely by utilizing the feature map of the rotated images, thus ensuring precise optimization and high image fidelity. Furthermore, we build a in-plane rotation focused benchmark called RotateBench, the first benchmark to evaluate the performance of point-based image editing method under in-plane rotation scenario on both real images and generated images. A thorough user study demonstrates the superior capability in accomplishing in-plane rotation that users intend to achieve, comparing the DragDiffusion baseline and other existing diffusion-based methods. See the project page https://github.com/Tony-Lowe/RotationDrag for code and experiment results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
