Although diffusion-based image virtual try-on has made considerable progress, emerging approaches still struggle to effectively address the issue of hand occlusion (i.e., clothing regions occluded by the hand part), leading to a notable degradation of the try-on performance. To tackle this issue widely existing in real-world scenarios, we propose VTON-HandFit, leveraging the power of hand priors to reconstruct the appearance and structure for hand occlusion cases. Firstly, we tailor a Handpose Aggregation Net using the ControlNet-based structure explicitly and adaptively encoding the global hand and pose priors. Besides, to fully exploit the hand-related structure and appearance information, we propose Hand-feature Disentanglement Embedding module to disentangle the hand priors into the hand structure-parametric and visual-appearance features, and customize a masked cross attention for further decoupled feature embedding. Lastly, we customize a hand-canny constraint loss to better learn the structure edge knowledge from the hand template of model image. VTON-HandFit outperforms the baselines in qualitative and quantitative evaluations on the public dataset and our self-collected hand-occlusion Handfit-3K dataset particularly for the arbitrary hand pose occlusion cases in real-world scenarios. The Code and dataset will be available at \url{https://github.com/VTON-HandFit/VTON-HandFit}.
The virtual try-on system has gained great attention due to its potential to give customers a realistic, personalized product presentation in virtualized settings. In this paper, we present PT-VTON, a novel pose-transfer-based framework for cloth transfer that enables virtual try-on with arbitrary poses. PT-VTON can be applied to the fashion industry within minimal modification of existing systems while satisfying the overall visual fashionability and detailed fabric appearance requirements. It enables efficient clothes transferring between model and user images with arbitrary pose and body sh…
Image-based virtual try-on techniques have shown great promise for enhancing the user-experience and improving customer satisfaction on fashion-oriented e-commerce platforms. However, existing techniques are currently still limited in the quality of the try-on results they are able to produce from input images of diverse characteristics. In this work, we propose a Context-Driven Virtual Try-On Network (C-VTON) that addresses these limitations and convincingly transfers selected clothing items to the target subjects even under challenging pose configurations and in the presence of self-occlusio…
FLDM-VTON: Faithful Latent Diffusion Model for Virtual Try-on
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（复旦大学脑智能科学与技术研究院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Suzhou Xiangji Technology Service Co., Ltd.（苏州湘记科技服务有限公司）
Image-based virtual try-on is challenging in fitting a target in-shop clothes into a reference person under diverse human poses. Previous works focus on preserving clothing details ( e.g., texture, logos, patterns ) when transferring desired clothes onto a target person under a fixed pose. However, the performances of existing methods significantly dropped when extending existing methods to multi-pose virtual try-on. In this paper, we propose an end-to-end Semantic Prediction Guidance multi-pose Virtual Try-On Network (SPG-VTON), which could fit the desired clothing into a reference person und…
Shenzhen Campus of SYSU（中山大学深圳校区）；UiT The Arctic University of Norway（挪威北极大学）；UC San Diego（加利福尼亚大学圣迭戈分校）；Momo（陌陌科技）
Virtual 3D try-on can provide an intuitive and realistic view for online shopping and has a huge potential commercial value. However, existing 3D virtual try-on methods mainly rely on annotated 3D human shapes and garment templates, which hinders their applications in practical scenarios. 2D virtual try-on approaches provide a faster alternative to manipulate clothed humans, but lack the rich and realistic 3D representation. In this paper, we propose a novel Monocular-to-3D Virtual Try-On Network (M3D-VTON) that builds on the merits of both 2D and 3D approaches. By integrating 2D information e…
Sun Yat-Sen University（中山大学）；UiT The Arctic University of Norway（挪威北极大学）；Momo Technology Company Limited（陌陌科技有限公司）；DarkMatter AI Research（暗物质人工智能研究院）
Despite recent progress on image-based virtual try-on, current methods are constraint by shared warping networks and thus fail to synthesize natural try-on results when faced with clothing categories that require different warping operations. In this paper, we address this problem by finding clothing category-specific warping networks for the virtual try-on task via Neural Architecture Search (NAS). We introduce a NAS-Warping Module and elaborately design a bilevel hierarchical search space to identify the optimal network-level and operation-level flow estimation architecture. Given the networ…
MOFA-VTON: More Fashion Possibilities with Fine-Grained Adaptations in Virtual Try-On
Harbin Institute of Technology（哈尔滨工业大学）；HiDream.ai Inc.（HiDream.ai公司）；Harbin Institute of Technology (Weihai) Qingdao Research Institute（哈尔滨工业大学（威海）青岛研究院）
MuGa-VTON: Multi-Garment Virtual Try-On via Diffusion Transformers with Prompt Customization
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Khalifa University（哈利法大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Jio Institute（Jio研究所）
Sun Yat-Sen University（中山大学）；ByteDance（字节跳动）；Carnegie Mellon University（卡内基梅隆大学）；Peng Cheng Laboratory（鹏城实验室）
Image-based Virtual Try-ON aims to transfer an in-shop garment onto a specific person. Existing methods employ a global warping module to model the anisotropic deformation for different garment parts, which fails to preserve the semantic information of different parts when receiving challenging inputs (e.g, intricate human poses, difficult garments). Moreover, most of them directly warp the input garment to align with the boundary of the preserved region, which usually requires texture squeezing to meet the boundary shape constraint and thus leads to texture distortion. The above inferior perf…
Undress to Redress: A Training-Free Framework for Virtual Try-On
Zhiying Li 1（李子莹1）；Junhao Wu 3（吴俊浩3）；Yeying Jin 4（金悦莹4）；Daiheng Gao 5（高代恒5）；Yun Ji 2（纪云2）；Kaichuan Kong 1（孔开传1）；Lei Yu 6（于磊6）；Hao Xu 7（徐浩7）；Kai Chen 7（陈凯7）；Bruce Gu 8（古布鲁斯8）；Nana Wang 9（王娜9）；Zhaoxin Fan 2（范朝新2）
Sun Yat-sen University（中山大学）；SenseTime Group Limited（商汤集团有限公司）
Image-based virtual try-on systems for fitting new in-shop clothes into a person image have attracted increasing research attention, yet is still challenging. A desirable pipeline should not only transform the target clothes into the most fitting shape seamlessly but also preserve well the clothes identity in the generated image, that is, the key characteristics (e.g. texture, logo, embroidery) that depict the original clothes. However, previous image-conditioned generation works fail to meet these critical requirements towards the plausible virtual try-on performance since they fail to handle…
Sun Yat-sen University（中山大学）；South China Normal University（华南师范大学）
Virtual try-on system under arbitrary human poses has huge application potential, yet raises quite a lot of challenges, e.g. self-occlusions, heavy misalignment among diverse poses, and diverse clothes textures. Existing methods aim at fitting new clothes into a person can only transfer clothes on the fixed human pose, but still show unsatisfactory performances which often fail to preserve the identity, lose the texture details, and decrease the diversity of poses. In this paper, we make the first attempt towards multi-pose guided virtual try-on system, which enables transfer clothes on a pers…
Significance of Skeleton-based Features in Virtual Try-On
TCG Centres for Research and Education in Science and Technology（TCG科学技术研究与教育中心）；Osaka University（大阪大学）；Indian Statistical Institute（印度统计研究所）；Indian Institute of Information Technology Kalyani（印度卡利亚尼信息技术学院）
