上下文内编辑：在大规模扩散Transformer中利用上下文内生成实现指令式图像编辑
In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
AI总结：
本文提出ICEdit方法，通过上下文内编辑、最小参数微调和早期过滤推理时扩展，仅用0.1%训练数据和1%可训练参数实现指令式图像编辑的最优性能。
摘要：
基于指令的图像编辑能够通过自然语言提示实现精确修改，但现有方法面临精度与效率的权衡：微调需要海量数据集（超过1000万）和大量计算资源，而无训练方法则存在指令理解能力较弱的问题。我们通过提出ICEdit来解决这一问题，该方法利用大规模扩散Transformer（DiTs）固有的理解与生成能力，并采用三项关键创新：（1）无需修改架构的上下文内编辑范式；（2）用于质量提升的最小参数高效微调；（3）早期过滤推理时扩展，利用视觉语言模型（VLMs）选择高质量噪声样本以提高效率。实验表明，与先前方法相比，ICEdit仅使用0.1%的训练数据和1%的可训练参数就实现了最先进的编辑性能。我们的方法为指令式图像编辑中平衡精度与效率建立了新范式。代码和演示可在https://river-zhang.github.io/ICEdit-gh-pages/找到。
英文摘要：
Instruction-based image editing enables precise modifications via natural language prompts, but existing methods face a precision-efficiency tradeoff: fine-tuning demands massive datasets (>10M) and computational resources, while training-free approaches suffer from weak instruction comprehension. We address this by proposing ICEdit, which leverages the inherent comprehension and generation abilities of large-scale Diffusion Transformers (DiTs) through three key innovations: (1) An in-context editing paradigm without architectural modifications; (2) Minimal parameter-efficient fine-tuning for quality improvement; (3) Early Filter Inference-Time Scaling, which uses VLMs to select high-quality noise samples for efficiency. Experiments show that ICEdit achieves state-of-the-art editing performance with only 0.1\% of the training data and 1\% trainable parameters compared to previous methods. Our approach establishes a new paradigm for balancing precision and efficiency in instructional image editing. Codes and demos can be found in https://river-zhang.github.io/ICEdit-gh-pages/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
