arXiv 2601.228382026-01-30cs.CV

神经服装试穿器：通过语义增强和控制扩散模型实现定制虚拟试穿

Neural Clothing Tryer: Customized Virtual Try-On via Semantic Enhancement and Controlling Diffusion Model

Guangdong University of Technology, Guangzhou 510006, China；Guangdong Provincial Key Laboratory of Intellectual Property \& Big Data, Guangzhou 510006, China；South China University of Technology, Guangzhou 510006, China；Genstoraige Technology (Beijing) Co., Ltd, Beijing 100000, China

Zhijing Yang, Weiwei Zhang, Mingliang Yang, Siyuan Peng, Yukai Shi, Junpeng Tan, Tianshui Chen, Liruo Zhong

备注：: Accepted by Expert Systems with Applications. 16 pages, 10 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出NCT框架，通过语义增强和控制扩散模型实现定制虚拟试穿，提升虚拟试穿体验的灵活性和参与度。

摘要：

本文旨在解决一个新颖的定制虚拟试穿（Cu-VTON）任务，使指定的衣物能够叠加到可以按外观、姿势和其他属性进行定制的模型上。与传统VTON任务相比，它使用户能够根据个人喜好定制数字虚拟形象，从而通过更大的灵活性和参与度提升虚拟试穿体验。为解决此任务，我们引入了神经服装试穿器（NCT）框架，该框架利用配备了语义增强和控制模块的先进扩散模型，以更好地保留衣物的语义特征和纹理细节，同时促进模型姿势和外观的灵活编辑。具体而言，NCT引入了一个语义增强模块，利用衣物的语义描述，并利用视觉-语言编码器学习跨模态对齐的特征。这些对齐的特征作为条件输入传递给扩散模型，以增强衣物语义的保留。然后，设计了一个语义控制模块，该模块以衣物图像、定制姿势图像和语义描述作为输入，以保持衣物细节的同时编辑模型的姿势、表情和各种属性。在开放可用基准上的广泛实验表明，所提出的NCT框架具有优越的性能。

英文摘要：

This work aims to address a novel Customized Virtual Try-ON (Cu-VTON) task, enabling the superimposition of a specified garment onto a model that can be customized in terms of appearance, posture, and additional attributes. Compared with traditional VTON task, it enables users to tailor digital avatars to their individual preferences, thereby enhancing the virtual fitting experience with greater flexibility and engagement. To address this task, we introduce a Neural Clothing Tryer (NCT) framework, which exploits the advanced diffusion models equipped with semantic enhancement and controlling modules to better preserve semantic characterization and textural details of the garment and meanwhile facilitating the flexible editing of the model's postures and appearances. Specifically, NCT introduces a semantic-enhanced module to take semantic descriptions of garments and utilizes a visual-language encoder to learn aligned features across modalities. The aligned features are served as condition input to the diffusion model to enhance the preservation of the garment's semantics. Then, a semantic controlling module is designed to take the garment image, tailored posture image, and semantic description as input to maintain garment details while simultaneously editing model postures, expressions, and various attributes. Extensive experiments on the open available benchmark demonstrate the superior performance of the proposed NCT framework.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。