多模态图像合成与编辑：生成式AI时代
Multimodal Image Synthesis and Editing: The Generative AI Era
AI总结：
该综述梳理生成式AI时代多模态图像合成与编辑的研究进展，按数据模态和模型类型分类，介绍方法、数据集、指标与实验，分析挑战并指明未来方向。
摘要：
由于现实世界中的信息以多种模态存在，多模态信息之间的有效交互与融合对计算机视觉和深度学习研究中多模态数据的创建与感知起着关键作用。凭借在建模多模态信息交互方面的卓越能力，多模态图像合成与编辑近年来成为热门研究方向。多模态引导无需为网络训练提供显式指导，而是为图像合成与编辑提供了直观灵活的手段。另一方面，该领域在多模态特征对齐、高分辨率图像合成、可靠的评估指标等方面也面临若干挑战。在本综述中，我们全面梳理了近期多模态图像合成与编辑的研究进展，并根据数据模态和模型类型构建了分类体系。我们首先介绍图像合成与编辑中的不同引导模态，随后根据模型类型详尽阐述多模态图像合成与编辑方法。之后，我们介绍了基准数据集、评估指标以及相应的实验结果。最后，我们针对当前研究挑战和未来研究的可能方向提出了见解。与本综述相关的项目可在https://github.com/fnzhan/Generative-AI 获取。
英文摘要：
As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer vision and deep learning research. With superb power in modeling the interaction among multimodal information, multimodal image synthesis and editing has become a hot research topic in recent years. Instead of providing explicit guidance for network training, multimodal guidance offers intuitive and flexible means for image synthesis and editing. On the other hand, this field is also facing several challenges in alignment of multimodal features, synthesis of high-resolution images, faithful evaluation metrics, etc. In this survey, we comprehensively contextualize the advance of the recent multimodal image synthesis and editing and formulate taxonomies according to data modalities and model types. We start with an introduction to different guidance modalities in image synthesis and editing, and then describe multimodal image synthesis and editing approaches extensively according to their model types. After that, we describe benchmark datasets and evaluation metrics as well as corresponding experimental results. Finally, we provide insights about the current research challenges and possible directions for future research. A project associated with this survey is available at https://github.com/fnzhan/Generative-AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
