DPDEdit：用于多模态时尚图像编辑的细节保留扩散模型
DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
AI总结：
针对时尚图像编辑中难以准确识别编辑区域和保留服装纹理细节的问题，提出基于潜在扩散模型的DPDEdit架构，通过Grounded-SAM定位区域并设计纹理注入与细化机制，在图像保真度和多模态一致性上优于现有方法。
摘要：
时尚图像编辑是设计师通过交互式可视化设计概念来传达创意想法的重要工具。当前的时尚图像编辑技术，尽管在多模态提示和强大的扩散模型方面取得了进展，但往往难以准确识别编辑区域并保留所需的服装纹理细节。为了解决这些挑战，我们引入了一种基于潜在扩散模型的新型多模态时尚图像编辑架构，称为细节保留扩散模型（DPDEdit）。DPDEdit通过整合文本提示、区域掩码、人体姿态图像和服装纹理图像来指导扩散模型的时尚图像生成。为了精确定位编辑区域，我们首先引入Grounded-SAM根据用户的文本描述预测编辑区域，然后将其与其他条件结合以进行局部编辑。为了将给定服装纹理的细节转移到目标时尚图像中，我们提出了一种纹理注入与细化机制。具体而言，该机制采用解耦的交叉注意力层来整合文本描述和纹理图像，并引入辅助U-Net以保留生成服装纹理的高频细节。此外，我们使用多模态大语言模型扩展了VITON-HD数据集，以生成带有纹理图像和文本描述的配对样本。大量实验表明，我们的DPDEdit在图像保真度以及与给定多模态输入的一致性方面优于现有最先进的方法。
英文摘要：
Fashion image editing is a crucial tool for designers to convey their creative ideas by visualizing design concepts interactively. Current fashion image editing techniques, though advanced with multimodal prompts and powerful diffusion models, often struggle to accurately identify editing regions and preserve the desired garment texture detail. To address these challenges, we introduce a new multimodal fashion image editing architecture based on latent diffusion models, called Detail-Preserved Diffusion Models (DPDEdit). DPDEdit guides the fashion image generation of diffusion models by integrating text prompts, region masks, human pose images, and garment texture images. To precisely locate the editing region, we first introduce Grounded-SAM to predict the editing region based on the user's textual description, and then combine it with other conditions to perform local editing. To transfer the detail of the given garment texture into the target fashion image, we propose a texture injection and refinement mechanism. Specifically, this mechanism employs a decoupled cross-attention layer to integrate textual descriptions and texture images, and incorporates an auxiliary U-Net to preserve the high-frequency details of generated garment texture. Additionally, we extend the VITON-HD dataset using a multimodal large language model to generate paired samples with texture images and textual descriptions. Extensive experiments show that our DPDEdit outperforms state-of-the-art methods in terms of image fidelity and coherence with the given multimodal inputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
