InstructGIE：迈向泛化的图像编辑
InstructGIE: Towards Generalizable Image Editing
AI总结：
提出InstructGIE框架，通过增强上下文学习、选择性区域匹配和语言统一技术，结合新数据集，实现图像编辑的高质量与跨任务泛化。
摘要：
图像编辑领域的最新进展得益于去噪扩散模型的发展，标志着该领域的一次重大飞跃。尽管取得了这些进展，近期图像编辑方法的泛化能力仍然受限。针对这一挑战，我们的研究提出了一种新颖的图像编辑框架，通过增强上下文学习能力和统一语言指令来提升泛化鲁棒性。该框架包含一个专门针对图像编辑任务优化的模块，利用VMamba块和编辑偏移匹配策略来增强上下文学习。此外，我们揭示了一种选择性区域匹配技术，专门用于处理和修正生成图像中的损坏细节（如人脸特征），以进一步提高质量。我们方法的另一个关键创新是集成了语言统一技术，该技术将语言嵌入与编辑语义对齐，以提升图像编辑的质量。此外，我们编制了首个包含视觉提示和编辑指令的图像编辑数据集，可用于增强上下文能力。基于该数据集训练后，我们的方法不仅为已训练任务实现了卓越的合成质量，而且通过定制提示在未见过的视觉任务中展示了强大的泛化能力。
英文摘要：
Recent advances in image editing have been driven by the development of denoising diffusion models, marking a significant leap forward in this field. Despite these advances, the generalization capabilities of recent image editing approaches remain constrained. In response to this challenge, our study introduces a novel image editing framework with enhanced generalization robustness by boosting in-context learning capability and unifying language instruction. This framework incorporates a module specifically optimized for image editing tasks, leveraging the VMamba Block and an editing-shift matching strategy to augment in-context learning. Furthermore, we unveil a selective area-matching technique specifically engineered to address and rectify corrupted details in generated images, such as human facial features, to further improve the quality. Another key innovation of our approach is the integration of a language unification technique, which aligns language embeddings with editing semantics to elevate the quality of image editing. Moreover, we compile the first dataset for image editing with visual prompts and editing instructions that could be used to enhance in-context capability. Trained on this dataset, our methodology not only achieves superior synthesis quality for trained tasks, but also demonstrates robust generalization capability across unseen vision tasks through tailored prompts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
