Multimodal Garment Designer：用于时尚图像编辑的以人为中心的潜在扩散模型
Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing
AI总结：
针对时尚设计中的图像编辑需求，提出基于潜在扩散模型的多模态条件编辑架构，扩展两个时尚数据集，实验验证其在真实感与输入一致性上的有效性。
摘要：
时尚插画被设计师用于传达设计构想，将设计理念从概念化推向落地，展现服装与人体的互动方式。在此背景下，计算机视觉可用于优化时尚设计流程。与以往主要聚焦服装虚拟试穿的研究不同，我们提出多模态条件时尚图像编辑任务，通过遵循文本、人体姿态、服装草图等多模态提示，引导生成以人为中心的时尚图像。我们提出一种基于潜在扩散模型的新架构来解决该问题，这一方法此前尚未在时尚领域得到应用。鉴于目前缺乏适用于该任务的数据集，我们还对半自动收集的多模态标注进行扩展，构建了两个新的时尚数据集Dress Code和VITON-HD。在这些新数据集上的实验结果表明，我们的方案在真实感以及与给定多模态输入的一致性方面均具备有效性。源代码与收集的多模态标注已公开于：https://github.com/aimagelab/multimodal-garment-designer。
英文摘要：
Fashion illustration is used by designers to communicate their vision and to bring the design idea from conceptualization to realization, showing how clothes interact with the human body. In this context, computer vision can thus be used to improve the fashion design process. Differently from previous works that mainly focused on the virtual try-on of garments, we propose the task of multimodal-conditioned fashion image editing, guiding the generation of human-centric fashion images by following multimodal prompts, such as text, human body poses, and garment sketches. We tackle this problem by proposing a new architecture based on latent diffusion models, an approach that has not been used before in the fashion domain. Given the lack of existing datasets suitable for the task, we also extend two existing fashion datasets, namely Dress Code and VITON-HD, with multimodal annotations collected in a semi-automatic manner. Experimental results on these new datasets demonstrate the effectiveness of our proposal, both in terms of realism and coherence with the given multimodal inputs. Source code and collected multimodal annotations are publicly available at: https://github.com/aimagelab/multimodal-garment-designer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
