利用即时注意力掩码实现高效的基于扩散的图像编辑
Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks
AI总结：
针对扩散图像编辑中掩码获取低效的问题，提出无需训练的InstDiffEdit，利用跨模态注意力即时生成精确掩码，并构建Editing-Mask基准，实验显示其质量更优且推理快5至6倍。
摘要：
基于扩散的图像编辑（DIE）是一个新兴的研究热点，它通常应用语义掩码来控制基于扩散编辑的目标区域。然而，现有大多数解决方案通过人工操作或离线处理来获取这些掩码，极大地降低了效率。本文中，我们针对文本到图像（T2I）扩散模型提出了一种新颖且高效的图像编辑方法，称为即时扩散编辑（InstDiffEdit）。具体而言，InstDiffEdit旨在利用现有扩散模型的跨模态注意力能力，在扩散步骤中实现即时掩码引导。为减少注意力图的噪声并实现完全自动化，我们为InstDiffEdit配备了一种无需训练的细化方案，以自适应聚合注意力分布，从而自动且准确地生成掩码。同时，为补充现有DIE评估，我们提出了一个名为Editing-Mask的新基准，用于检验现有方法的掩码准确性和局部编辑能力。为验证InstDiffEdit，我们还在ImageNet和Imagen上开展了大量实验，并将其与一系列SOTA方法进行比较。实验结果表明，InstDiffEdit不仅在图像质量和编辑结果上均优于SOTA方法，而且具有快得多的推理速度，即快5至6倍。
英文摘要：
Diffusion-based Image Editing (DIE) is an emerging research hot-spot, which often applies a semantic mask to control the target area for diffusion-based editing. However, most existing solutions obtain these masks via manual operations or off-line processing, greatly reducing their efficiency. In this paper, we propose a novel and efficient image editing method for Text-to-Image (T2I) diffusion models, termed Instant Diffusion Editing(InstDiffEdit). In particular, InstDiffEdit aims to employ the cross-modal attention ability of existing diffusion models to achieve instant mask guidance during the diffusion steps. To reduce the noise of attention maps and realize the full automatics, we equip InstDiffEdit with a training-free refinement scheme to adaptively aggregate the attention distributions for the automatic yet accurate mask generation. Meanwhile, to supplement the existing evaluations of DIE, we propose a new benchmark called Editing-Mask to examine the mask accuracy and local editing ability of existing methods. To validate InstDiffEdit, we also conduct extensive experiments on ImageNet and Imagen, and compare it with a bunch of the SOTA methods. The experimental results show that InstDiffEdit not only outperforms the SOTA methods in both image quality and editing results, but also has a much faster inference speed, i.e., +5 to +6 times.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
