Focus on Your Instruction：基于注意力调制的细粒度多指令图像编辑
Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation
AI总结：
针对现有扩散模型指令图像编辑易改动非目标区域、多指令编辑效果差的问题，提出FoI方法，通过挖掘IP2P隐式定位能力提取指令对应区域，结合交叉注意力调制与掩码引导解耦采样实现精准编辑，无需额外训练，多指令编辑性能优于现有方法。
摘要：
近年来，基于扩散模型的方法（如InstructPix2Pix (IP2P)）已实现了有效的指令式图像编辑，仅需用户提供自然语言指令即可完成操作。然而，这类方法往往会无意中改动非目标区域，且在多指令编辑任务中表现不佳，导致编辑效果受损。为解决这些问题，我们提出了Focus on Your Instruction（FoI，聚焦你的指令）方法，该方法无需额外训练或测试阶段优化，即可实现多指令下精准、协调的编辑效果。FoI主要围绕两个核心方向设计：（1）精准提取每条指令对应的感兴趣区域；（2）引导去噪过程集中在这些感兴趣区域内执行。针对第一个目标，我们从指令与图像的交叉注意力中挖掘出IP2P的隐式定位能力，并据此开发了一种有效的掩码提取方法。针对第二个目标，我们引入了交叉注意力调制模块，初步实现目标编辑区域与无关区域的隔离。此外，我们还提出了掩码引导的解耦采样策略，进一步确保区域隔离的清晰性。实验结果表明，FoI在定量和定性评估中均优于现有方法，尤其在多指令编辑任务上表现突出。
英文摘要：
Recently, diffusion-based methods, like InstructPix2Pix (IP2P), have achieved effective instruction-based image editing, requiring only natural language instructions from the user. However, these methods often inadvertently alter unintended areas and struggle with multi-instruction editing, resulting in compromised outcomes. To address these issues, we introduce the Focus on Your Instruction (FoI), a method designed to ensure precise and harmonious editing across multiple instructions without extra training or test-time optimization. In the FoI, we primarily emphasize two aspects: (1) precisely extracting regions of interest for each instruction and (2) guiding the denoising process to concentrate within these regions of interest. For the first objective, we identify the implicit grounding capability of IP2P from the cross-attention between instruction and image, then develop an effective mask extraction method. For the second objective, we introduce a cross attention modulation module for rough isolation of target editing regions and unrelated regions. Additionally, we introduce a mask-guided disentangle sampling strategy to further ensure clear region isolation. Experimental results demonstrate that FoI surpasses existing methods in both quantitative and qualitative evaluations, especially excelling in multi-instruction editing task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
