Magic Clothing：可控的服装驱动图像合成
Magic Clothing: Controllable Garment-Driven Image Synthesis
AI总结：
提出基于潜在扩散模型的Magic Clothing架构，通过服装提取器和自注意力融合实现可控的服装驱动图像合成，支持多扩展并达到SOTA效果。
摘要：
我们提出了Magic Clothing，一种基于潜在扩散模型（LDM）的网络架构，用于探索尚未充分研究的服装驱动图像合成任务。该任务旨在生成穿着目标服装的定制化角色，并配以多样化的文本提示。在此过程中，图像可控性是最关键的问题，即保留服装细节并保持对文本提示的忠实度。为此，我们引入了一个服装提取器来捕捉详细的服装特征，并采用自注意力融合将其整合到预训练的LDM中，确保目标角色身上的服装细节保持不变。随后，我们利用联合无分类器指导来平衡服装特征和文本提示对生成结果的控制。同时，所提出的服装提取器是一个即插即用模块，适用于各种微调的LDM，并且可以与ControlNet和IP-Adapter等其他扩展结合，以增强生成角色的多样性和可控性。此外，我们设计了Matched-Points-LPIPS（MP-LPIPS），这是一种用于评估目标图像与源服装一致性的鲁棒指标。大量实验表明，我们的Magic Clothing在服装驱动图像合成的各种条件控制下均取得了最先进的结果。源代码可在 https://github.com/ShineChen1024/MagicClothing 获取。
英文摘要：
We propose Magic Clothing, a latent diffusion model (LDM)-based network architecture for an unexplored garment-driven image synthesis task. Aiming at generating customized characters wearing the target garments with diverse text prompts, the image controllability is the most critical issue, i.e., to preserve the garment details and maintain faithfulness to the text prompts. To this end, we introduce a garment extractor to capture the detailed garment features, and employ self-attention fusion to incorporate them into the pretrained LDMs, ensuring that the garment details remain unchanged on the target character. Then, we leverage the joint classifier-free guidance to balance the control of garment features and text prompts over the generated results. Meanwhile, the proposed garment extractor is a plug-in module applicable to various finetuned LDMs, and it can be combined with other extensions like ControlNet and IP-Adapter to enhance the diversity and controllability of the generated characters. Furthermore, we design Matched-Points-LPIPS (MP-LPIPS), a robust metric for evaluating the consistency of the target image to the source garment. Extensive experiments demonstrate that our Magic Clothing achieves state-of-the-art results under various conditional controls for garment-driven image synthesis. Our source code is available at https://github.com/ShineChen1024/MagicClothing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
