DreamMatcher：用于语义一致文本到图像个性化的外观匹配自注意力
DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization
AI总结：
提出DreamMatcher插件方法，通过语义匹配替换目标值并引入语义一致掩码策略，在保留预训练模型结构生成能力的同时实现准确的文本到图像个性化外观模仿。
摘要：
文本到图像（T2I）个性化的目标是将扩散模型定制为用户提供的参考概念，生成与目标提示对齐的多样化概念图像。使用唯一文本嵌入表示参考概念的传统方法往往无法准确模仿参考的外观。为解决此问题，一种方案是将参考图像显式地条件化到目标去噪过程中，即键值替换。然而，先前的方法受限于局部编辑，因为它们破坏了预训练T2I模型的结构路径。为克服此问题，我们提出一种新颖的插件方法DreamMatcher，将T2I个性化重新表述为语义匹配。具体而言，DreamMatcher通过语义匹配对齐的参考值替换目标值，同时保持结构路径不变，以保留预训练T2I模型生成多样结构的通用能力。我们还引入了语义一致掩码策略，将个性化概念与目标提示引入的不相关区域隔离。DreamMatcher与现有T2I模型兼容，在复杂场景中表现出显著改进。深入分析证明了该方法的有效性。
英文摘要：
The objective of text-to-image (T2I) personalization is to customize a diffusion model to a user-provided reference concept, generating diverse images of the concept aligned with the target prompts. Conventional methods representing the reference concepts using unique text embeddings often fail to accurately mimic the appearance of the reference. To address this, one solution may be explicitly conditioning the reference images into the target denoising process, known as key-value replacement. However, prior works are constrained to local editing since they disrupt the structure path of the pre-trained T2I model. To overcome this, we propose a novel plug-in method, called DreamMatcher, which reformulates T2I personalization as semantic matching. Specifically, DreamMatcher replaces the target values with reference values aligned by semantic matching, while leaving the structure path unchanged to preserve the versatile capability of pre-trained T2I models for generating diverse structures. We also introduce a semantic-consistent masking strategy to isolate the personalized concept from irrelevant regions introduced by the target prompts. Compatible with existing T2I models, DreamMatcher shows significant improvements in complex scenarios. Intensive analyses demonstrate the effectiveness of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
