Fashion-RAG：基于检索增强生成的多模态时尚图像编辑
Fashion-RAG: Multimodal Fashion Image Editing via Retrieval-Augmented Generation
AI总结：
针对现有虚拟试穿依赖具体服装输入、难以满足用户仅凭文本定制需求的问题，提出Fashion-RAG方法，通过检索匹配服装并利用文本反演集成到扩散模型生成中，在Dress Code数据集上超越现有方法，实现多模态时尚图像编辑。
摘要：
近年来，随着电子商务平台和虚拟应用的普及，时尚产业越来越多地采用人工智能技术来提升客户体验。在各种任务中，虚拟试穿和多模态时尚图像编辑——利用文本、服装草图、人体姿态等多种输入模态——已成为一个关键研究领域。扩散模型已成为此类生成任务的主流方法，提供优越的图像质量和多样性。然而，现有的大多数虚拟试穿方法依赖于特定的服装输入，这在现实场景中往往不切实际，因为用户可能仅提供文本描述。为解决这一局限，本工作引入了时尚检索增强生成（Fashion-RAG），一种新颖的方法，能够根据用户以文本形式提供的偏好定制时尚单品。我们的方法检索多个与输入描述匹配的服装，并通过整合检索到的单品属性生成个性化图像。为此，我们采用文本反演技术，将检索到的服装图像投影到Stable Diffusion文本编码器的文本嵌入空间中，使检索元素能够无缝集成到生成过程中。在Dress Code数据集上的实验结果表明，Fashion-RAG在定性和定量上均优于现有方法，有效捕捉检索服装的细粒度视觉细节。据我们所知，这是首个专门针对多模态时尚图像编辑引入检索增强生成方法的工作。
英文摘要：
In recent years, the fashion industry has increasingly adopted AI technologies to enhance customer experience, driven by the proliferation of e-commerce platforms and virtual applications. Among the various tasks, virtual try-on and multimodal fashion image editing -- which utilizes diverse input modalities such as text, garment sketches, and body poses -- have become a key area of research. Diffusion models have emerged as a leading approach for such generative tasks, offering superior image quality and diversity. However, most existing virtual try-on methods rely on having a specific garment input, which is often impractical in real-world scenarios where users may only provide textual specifications. To address this limitation, in this work we introduce Fashion Retrieval-Augmented Generation (Fashion-RAG), a novel method that enables the customization of fashion items based on user preferences provided in textual form. Our approach retrieves multiple garments that match the input specifications and generates a personalized image by incorporating attributes from the retrieved items. To achieve this, we employ textual inversion techniques, where retrieved garment images are projected into the textual embedding space of the Stable Diffusion text encoder, allowing seamless integration of retrieved elements into the generative process. Experimental results on the Dress Code dataset demonstrate that Fashion-RAG outperforms existing methods both qualitatively and quantitatively, effectively capturing fine-grained visual details from retrieved garments. To the best of our knowledge, this is the first work to introduce a retrieval-augmented generation approach specifically tailored for multimodal fashion image editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
