DiffUHaul：一种无需训练的图像中物体拖拽方法
DiffUHaul: A Training-Free Method for Object Dragging in Images
AI总结：
DiffUHaul提出一种无需训练的物体拖拽方法，利用局部化文本到图像模型的空间理解，通过注意力掩码、自注意力共享和扩散锚定技术实现高质量编辑，并引入自动化评估流程验证效果。
摘要：
文本到图像的扩散模型已被证明在解决许多图像编辑任务方面是有效的。然而，在场景内无缝重新定位物体这一看似简单的任务仍然出人意料地具有挑战性。现有解决该问题的方法由于缺乏空间推理能力，常常难以在现实世界场景中可靠地工作。在这项工作中，我们提出了一种无需训练的方法，称为DiffUHaul，它利用局部化文本到图像模型的空间理解能力来完成物体拖拽任务。由于模型中物体表示的内在纠缠，盲目地操作局部化模型的布局输入往往会导致较低的编辑性能。为此，我们首先在每个去噪步骤中应用注意力掩码，使生成过程在不同物体之间更加解耦，并采用自注意力共享机制来保留高层次的物体外观。此外，我们提出了一种新的扩散锚定技术：在早期的去噪步骤中，我们在源图像和目标图像之间插值注意力特征，以平滑地将新布局与原始外观融合；在晚期的去噪步骤中，我们将源图像的局部化特征传递到插值图像中，以保留细粒度的物体细节。为了使DiffUHaul适应真实图像编辑，我们应用了DDPM自注意力分桶技术，该技术可以更好地用局部化模型重建真实图像。最后，我们为该任务引入了一个自动化评估流程，并展示了我们方法的有效性。我们的结果通过用户偏好研究得到了进一步证实。
英文摘要：
Text-to-image diffusion models have proven effective for solving many image editing tasks. However, the seemingly straightforward task of seamlessly relocating objects within a scene remains surprisingly challenging. Existing methods addressing this problem often struggle to function reliably in real-world scenarios due to lacking spatial reasoning. In this work, we propose a training-free method, dubbed DiffUHaul, that harnesses the spatial understanding of a localized text-to-image model, for the object dragging task. Blindly manipulating layout inputs of the localized model tends to cause low editing performance due to the intrinsic entanglement of object representation in the model. To this end, we first apply attention masking in each denoising step to make the generation more disentangled across different objects and adopt the self-attention sharing mechanism to preserve the high-level object appearance. Furthermore, we propose a new diffusion anchoring technique: in the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance; in the later denoising steps, we pass the localized features from the source images to the interpolated images to retain fine-grained object details. To adapt DiffUHaul to real-image editing, we apply a DDPM self-attention bucketing that can better reconstruct real images with the localized model. Finally, we introduce an automated evaluation pipeline for this task and showcase the efficacy of our method. Our results are reinforced through a user preference study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
