InstructUDrag：结合文本指令与对象拖拽的交互式图像编辑
InstructUDrag: Joint Text Instructions and Object Dragging for Interactive Image Editing
AI总结：
提出InstructUDrag扩散框架，结合文本指令与对象拖拽，通过双分支协同实现精确对象移动与语义编辑，实验证明其高保真与灵活性。
摘要：
文本到图像的扩散模型在图像编辑方面展现出巨大潜力，基于文本和对象拖拽的方法已成为关键技术途径。然而，每种方法都有其固有的局限性：基于文本的方法难以实现精确的对象定位，而对象拖拽方法则局限于静态的重新定位。为解决这些问题，我们提出了InstructUDrag，一个基于扩散的框架，将文本指令与对象拖拽相结合，实现对象拖拽与基于文本的图像编辑的同步进行。我们的框架将对象拖拽视为一个图像重建过程，分为两个协同分支。移动重建分支利用基于能量的梯度引导来精确移动对象，并通过细化交叉注意力图来提高重新定位的精度。文本驱动编辑分支与重建分支共享梯度信号，确保变换的一致性，并允许对对象属性进行细粒度的控制。我们还采用了DDPM反演，并将先验信息注入噪声图，以保持被移动对象的结构。大量实验表明，InstructUDrag促进了灵活、高保真的图像编辑，在对象重新定位的精确性和对图像内容的语义控制方面均表现出色。
英文摘要：
Text-to-image diffusion models have shown great potential for image editing, with techniques such as text-based and object-dragging methods emerging as key approaches. However, each of these methods has inherent limitations: text-based methods struggle with precise object positioning, while object dragging methods are confined to static relocation. To address these issues, we propose InstructUDrag, a diffusion-based framework that combines text instructions with object dragging, enabling simultaneous object dragging and text-based image editing. Our framework treats object dragging as an image reconstruction process, divided into two synergistic branches. The moving-reconstruction branch utilizes energy-based gradient guidance to move objects accurately, refining cross-attention maps to enhance relocation precision. The text-driven editing branch shares gradient signals with the reconstruction branch, ensuring consistent transformations and allowing fine-grained control over object attributes. We also employ DDPM inversion and inject prior information into noise maps to preserve the structure of moved objects. Extensive experiments demonstrate that InstructUDrag facilitates flexible, high-fidelity image editing, offering both precision in object relocation and semantic control over image content.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
