FreeCustom：面向多概念组合的无调优定制图像生成
FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition
AI总结：
提出无调优方法FreeCustom，基于每概念单张参考图像，通过多参考自注意力机制与加权掩码策略实现多概念组合定制图像生成，性能优于或媲美基于训练的方法。
摘要：
得益于大规模预训练的文本到图像（T2I）生成模型，定制图像生成领域取得了显著进展，该任务旨在生成用户指定的概念。现有方法主要集中于单概念定制，在处理涉及多概念组合的复杂场景时仍面临挑战。这些方法通常需要使用少量图像进行重新训练或微调，导致训练过程耗时且阻碍其快速部署。此外，依赖多张图像来表示单一概念增加了定制的难度。为此，我们提出了FreeCustom，一种新颖的无调优方法，基于参考概念生成多概念组合的定制图像，每个概念仅需一张输入图像。具体而言，我们引入了一种新的多参考自注意力（MRSA）机制和加权掩码策略，使生成的图像能够访问并更关注参考概念。此外，MRSA利用了我们的关键发现：当提供具有上下文交互的图像时，输入概念能更好地被保留。实验表明，我们方法生成的图像与给定概念一致，并与输入文本对齐更好。我们的方法在多概念组合和单概念定制方面优于或与基于训练的方法相当，但更简单。代码可在https://github.com/aim-uofa/FreeCustom获取。
英文摘要：
Benefiting from large-scale pre-trained text-to-image (T2I) generative models, impressive progress has been achieved in customized image generation, which aims to generate user-specified concepts. Existing approaches have extensively focused on single-concept customization and still encounter challenges when it comes to complex scenarios that involve combining multiple concepts. These approaches often require retraining/fine-tuning using a few images, leading to time-consuming training processes and impeding their swift implementation. Furthermore, the reliance on multiple images to represent a singular concept increases the difficulty of customization. To this end, we propose FreeCustom, a novel tuning-free method to generate customized images of multi-concept composition based on reference concepts, using only one image per concept as input. Specifically, we introduce a new multi-reference self-attention (MRSA) mechanism and a weighted mask strategy that enables the generated image to access and focus more on the reference concepts. In addition, MRSA leverages our key finding that input concepts are better preserved when providing images with context interactions. Experiments show that our method's produced images are consistent with the given concepts and better aligned with the input text. Our method outperforms or performs on par with other training-based methods in terms of multi-concept composition and single-concept customization, but is simpler. Codes can be found at https://github.com/aim-uofa/FreeCustom.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
