基于文本到图像扩散模型的多模态引导图像编辑综述
A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models
AI总结：
该综述全面梳理了基于T2I扩散模型的多模态引导图像编辑技术，界定研究范围，提出统一算法框架并分析组件特性，讨论训练类方法与视频编辑应用，最后指出领域挑战与未来方向。
摘要：
图像编辑旨在对给定的合成或真实图像进行编辑，以满足用户的特定需求。作为人工智能生成内容（AIGC）领域中一个极具前景又充满挑战的方向，图像编辑近年来得到了广泛研究。该领域近期的重大进展依托于文本到图像（text-to-image, T2I）扩散模型的发展，这类模型可根据文本提示生成图像，展现出卓越的生成能力，已成为图像编辑领域广泛使用的工具。基于T2I的图像编辑方法显著提升了编辑性能，还为用户提供了友好的交互界面，支持通过多模态输入引导内容修改。\n本综述对利用T2I扩散模型的多模态引导图像编辑技术展开全面梳理。首先，我们从整体视角界定了图像编辑的研究范围，详细介绍了各类控制信号与编辑场景。随后，我们提出一个统一框架对编辑过程进行形式化划分，将其归为两大主要算法家族，该框架为用户实现特定编辑目标提供了设计空间。接着，我们对框架内的各个组件进行深入分析，探究不同组件组合的特点与适用场景。鉴于基于训练的方法是在用户引导下学习源图像到目标图像的直接映射，我们对这类方法进行了单独讨论，并介绍了不同场景下源图像的注入方案。此外，我们还梳理了二维技术在视频编辑中的应用，重点介绍了帧间不一致问题的解决方案。最后，我们讨论了该领域尚存的开放挑战，并提出了潜在的未来研究方向。我们会在https://github.com/xinchengshuai/Awesome-Image-Editing 持续追踪相关研究工作。
英文摘要：
Image editing aims to edit the given synthetic or real image to meet the specific requirements from users. It is widely studied in recent years as a promising and challenging field of Artificial Intelligence Generative Content (AIGC). Recent significant advancement in this field is based on the development of text-to-image (T2I) diffusion models, which generate images according to text prompts. These models demonstrate remarkable generative capabilities and have become widely used tools for image editing. T2I-based image editing methods significantly enhance editing performance and offer a user-friendly interface for modifying content guided by multimodal inputs. In this survey, we provide a comprehensive review of multimodal-guided image editing techniques that leverage T2I diffusion models. First, we define the scope of image editing from a holistic perspective and detail various control signals and editing scenarios. We then propose a unified framework to formalize the editing process, categorizing it into two primary algorithm families. This framework offers a design space for users to achieve specific goals. Subsequently, we present an in-depth analysis of each component within this framework, examining the characteristics and applicable scenarios of different combinations. Given that training-based methods learn to directly map the source image to target one under user guidance, we discuss them separately, and introduce injection schemes of source image in different scenarios. Additionally, we review the application of 2D techniques to video editing, highlighting solutions for inter-frame inconsistency. Finally, we discuss open challenges in the field and suggest potential future research directions. We keep tracing related works at https://github.com/xinchengshuai/Awesome-Image-Editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
