用于时尚图像编辑的多模态条件潜在扩散模型
Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing
AI总结：
本文提出扩展潜在扩散模型以处理多模态条件时尚图像编辑任务，通过修改去噪网络结构并引入文本反演技术融合文本、姿势、草图和面料纹理，并在扩展的 Dress Code 和 VITON-HD 数据集上验证了方法的有效性。
摘要：
时尚插画是设计师传达其创意愿景并将设计概念转化为展示服装与人体之间相互作用的具象表达的重要媒介。在时尚设计背景下，计算机视觉技术有潜力增强并简化设计流程。不同于以往主要专注于虚拟试穿的研究，本文着手处理多模态条件时尚图像编辑任务。我们的方法旨在生成以人为中心的时尚图像，其由多模态提示引导，包括文本、人体姿势、服装草图和面料纹理。为解决这一问题，我们提出扩展潜在扩散模型以纳入这些多模态，并修改去噪网络的结构，将多模态提示作为输入。为了使所提出的架构以面料纹理为条件，我们采用文本反演技术，并让去噪网络的不同交叉注意力层关注文本和纹理信息，从而纳入不同粒度的条件细节。鉴于缺乏用于该任务的数据集，我们扩展了两个现有的时尚数据集 Dress Code 和 VITON-HD，为其添加了多模态标注。实验评估证明了我们提出的方法在真实感以及与所提供多模态输入的连贯性方面的有效性。
英文摘要：
Fashion illustration is a crucial medium for designers to convey their creative vision and transform design concepts into tangible representations that showcase the interplay between clothing and the human body. In the context of fashion design, computer vision techniques have the potential to enhance and streamline the design process. Departing from prior research primarily focused on virtual try-on, this paper tackles the task of multimodal-conditioned fashion image editing. Our approach aims to generate human-centric fashion images guided by multimodal prompts, including text, human body poses, garment sketches, and fabric textures. To address this problem, we propose extending latent diffusion models to incorporate these multiple modalities and modifying the structure of the denoising network, taking multimodal prompts as input. To condition the proposed architecture on fabric textures, we employ textual inversion techniques and let diverse cross-attention layers of the denoising network attend to textual and texture information, thus incorporating different granularity conditioning details. Given the lack of datasets for the task, we extend two existing fashion datasets, Dress Code and VITON-HD, with multimodal annotations. Experimental evaluations demonstrate the effectiveness of our proposed approach in terms of realism and coherence concerning the provided multimodal inputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
