解耦扩散Transformer中的指令影响以实现并行多指令引导的图像编辑
Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing
AI总结：
本文提出指令影响解耦（IID）框架，通过分析扩散Transformer的自注意力机制生成指令特定掩码，在单个去噪过程中并行执行多指令图像编辑，减少扩散步骤并提升保真度与指令完成度。
摘要：
指令引导的图像编辑使用户能够使用自然语言指定修改，提供了更大的灵活性和控制力。在现有框架中，扩散Transformer（DiTs）在可扩展性和性能上优于基于U-Net的扩散模型。然而，尽管现实场景通常需要同时执行多个指令，逐步编辑会遭受累积误差和质量下降的问题，而将多个指令整合到单一提示中通常由于指令冲突导致编辑不完整。我们提出指令影响解耦（Instruction Influence Disentanglement, IID），这是一种新颖的框架，能够在单个去噪过程中并行执行多个指令，专为基于DiT的模型设计。通过分析DiT中的自注意力机制，我们识别了多指令设置下独特的注意力模式，并推导出指令特定的注意力掩码，以解耦每条指令的影响。这些掩码引导编辑过程，确保局部修改的同时保持未编辑区域的一致性。在开源和自定义数据集上的大量实验表明，与现有基线相比，IID减少了扩散步骤，同时提高了保真度和指令完成度。代码将在论文被接收后公开发布。
英文摘要：
Instruction-guided image editing enables users to specify modifications using natural language, offering more flexibility and control. Among existing frameworks, Diffusion Transformers (DiTs) outperform U-Net-based diffusion models in scalability and performance. However, while real-world scenarios often require concurrent execution of multiple instructions, step-by-step editing suffers from accumulated errors and degraded quality, and integrating multiple instructions with a single prompt usually results in incomplete edits due to instruction conflicts. We propose Instruction Influence Disentanglement (IID), a novel framework enabling parallel execution of multiple instructions in a single denoising process, designed for DiT-based models. By analyzing self-attention mechanisms in DiTs, we identify distinctive attention patterns in multi-instruction settings and derive instruction-specific attention masks to disentangle each instruction's influence. These masks guide the editing process to ensure localized modifications while preserving consistency in non-edited regions. Extensive experiments on open-source and custom datasets demonstrate that IID reduces diffusion steps while improving fidelity and instruction completion compared to existing baselines. The codes will be publicly released upon the acceptance of the paper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
