A virtual try-on method takes a product image and an image of a model and produces an image of the model wearing the product. Most methods essentially compute warps from the product image to the model image and combine using image generation methods. However, obtaining a realistic image is challenging because the kinematics of garments is complex and because outline, texture, and shading cues in the image reveal errors to human viewers. The garment must have appropriate drapes; texture must be warped to be consistent with the shape of a draped garment; small details (buttons, collars, lapels, pockets, etc.) must be placed appropriately on the garment, and so on. Evaluation is particularly difficult and is usually qualitative.
This paper uses quantitative evaluation on a challenging, novel dataset to demonstrate that (a) for any warping method, one can choose target models automatically to improve results, and (b) learning multiple coordinated specialized warpers offers further improvements on results. Target models are chosen by a learned embedding procedure that predicts a representation of the products the model is wearing. This prediction is used to match products to models. Specialized warpers are trained by a method that encourages a second warper to perform well in locations where the first works poorly. The warps are then combined using a U-Net. Qualitative evaluation confirms that these improvements are wholesale over outline, texture shading, and garment details.
Previous virtual try-on methods usually focus on aligning a clothing item with a person, limiting their ability to exploit the complex pose, shape and skin color of the person, as well as the overall structure of the clothing, which is vital to photo-realistic virtual try-on. To address this potential weakness, we propose a fill in fabrics (FIFA) model, a self-supervised conditional generative adversarial network based framework comprised of a Fabricator and a unified virtual try-on pipeline with a Segmenter, Warper and Fuser. The Fabricator aims to reconstruct the clothing image when provided…
South China University of Technology（华南理工大学）；Nanyang Technological University（南洋理工大学）；Guangdong University of Technology（广东工业大学）
With the development of Generative Adversarial Network, image-based virtual try-on methods have made great progress. However, limited work has explored the task of video-based virtual try-on while it is important in real-world applications. Most existing video-based virtual try-on methods usually require clothing templates and they can only generate blurred and low-resolution results. To address these challenges, we propose a Memory-based Video virtual Try-On Network (MV-TON), which seamlessly transfers desired clothes to a target person without using any clothing templates and generates high-…
The University of Tokyo（东京大学）；The International Research Center for Neurointelligence（国际神经智能研究中心）
Generating a virtual try-on image from in-shop clothing images and a model person's snapshot is a challenging task because the human body and clothes have high flexibility in their shapes. In this paper, we develop a Virtual Try-on Generative Adversarial Network (VITON-GAN), that generates virtual try-on images using images of in-shop clothing and a model person. This method enhances the quality of the generated image when occlusion is present in a model person's image (e.g., arms crossed in front of the clothes) by adding an adversarial mechanism in the training pipeline.
Sun Yat-sen University（中山大学）；South China Normal University（华南师范大学）
Virtual try-on system under arbitrary human poses has huge application potential, yet raises quite a lot of challenges, e.g. self-occlusions, heavy misalignment among diverse poses, and diverse clothes textures. Existing methods aim at fitting new clothes into a person can only transfer clothes on the fixed human pose, but still show unsatisfactory performances which often fail to preserve the identity, lose the texture details, and decrease the diversity of poses. In this paper, we make the first attempt towards multi-pose guided virtual try-on system, which enables transfer clothes on a pers…
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Carnegie Mellon University（卡内基梅隆大学）；UiT The Arctic University of Norway（挪威北极大学）；ByteDance（字节跳动）
Image-based virtual try-on is one of the most promising applications of human-centric image generation due to its tremendous real-world potential. In this work, we take a step forwards to explore versatile virtual try-on solutions, which we argue should possess three main properties, namely, they should support unsupervised training, arbitrary garment categories, and controllable garment editing. To this end, we propose a characteristic-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN++ (PASTA-GAN++), to achieve a versatile system for high-resolution unpaired virtual try-…
Multi-pose virtual try-on (MPVTON) aims to fit a target garment onto a person at a target pose. Compared to traditional virtual try-on (VTON) that fits the garment but keeps the pose unchanged, MPVTON provides a better try-on experience, but is also more challenging due to the dual garment and pose editing objectives. Existing MPVTON methods adopt a pipeline comprising three disjoint modules including a target semantic layout prediction module, a coarse try-on image generator and a refinement try-on image generator. These models are trained separately, leading to sub-optimal model training and…
DiffusionTrend: A Minimalist Approach to Virtual Fashion Try-On
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）；Skywork AI, Singapore（新加坡Skywork AI）
The School of Electrical and Information Engineering, Tianjin University（天津大学电气自动化与信息工程学院）；The College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The School of Computing, National University of Singapore（新加坡国立大学计算机学院）
FDM-MFVT: Few-step Sampling Diffusion Model for Mask-Free Virtual Try-On
School of Electronic Information Engineering, Beihang University, Beijing, China（北京航空航天大学电子信息工程学院）；Zhongguancun Academy, Beijing, China（中关村学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China（虚拟现实技术与系统国家重点实验室）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We propose a flexible person generation framework called Dressing in Order (DiOr), which supports 2D pose transfer, virtual try-on, and several fashion editing tasks. The key to DiOr is a novel recurrent generation pipeline to sequentially put garments on a person, so that trying on the same garments in different orders will result in different looks. Our system can produce dressing effects not achievable by existing work, including different interactions of garments (e.g., wearing a top tucked into the bottom or over it), as well as layering of multiple garments of the same type (e.g., jacket…
Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image. Existing methods usually build up multi-stage frameworks to deal with clothes warping and body blending respectively, or rely heavily on intermediate parser-based labels which may be noisy or even inaccurate. To solve the above challenges, we propose a single-stage try-on framework by developing a novel Deformable Attention Flow (DAFlow), which applies the deformable attention scheme to multi-flow estimation. With pose keypoints as the guidance only, the self- and cross-defor…
Neural Clothing Tryer: Customized Virtual Try-On via Semantic Enhancement and Controlling Diffusion Model
Guangdong University of Technology, Guangzhou 510006, China；Guangdong Provincial Key Laboratory of Intellectual Property \& Big Data, Guangzhou 510006, China；South China University of Technology, Guangzhou 510006, China；Genstoraige Technology (Beijing) Co., Ltd, Beijing 100000, China
L’Oréal Research and Innovation（欧莱雅研究与创新部）；Télécom Paris, Institut Polytechnique de Paris（巴黎理工学院电信巴黎）；Modiface；Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）
Augmented reality applications have rapidly spread across online platforms, allowing consumers to virtually try-on a variety of products, such as makeup, hair dying, or shoes. However, parametrizing a renderer to synthesize realistic images of a given product remains a challenging task that requires expert knowledge. While recent work has introduced neural rendering methods for virtual try-on from example images, current approaches are based on large generative models that cannot be used in real-time on mobile devices. This calls for a hybrid method that combines the advantages of computer gra…
Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区）；UiT The Arctic University of Norway（挪威北极大学）；Peng Cheng Laboratory（鹏城实验室）
Image-based virtual try-on is one of the most promising applications of human-centric image generation due to its tremendous real-world potential. Yet, as most try-on approaches fit in-shop garments onto a target person, they require the laborious and restrictive construction of a paired training dataset, severely limiting their scalability. While a few recent works attempt to transfer garments directly from one person to another, alleviating the need to collect paired datasets, their performance is impacted by the lack of paired (supervised) information. In particular, disentangling style and…
FLDM-VTON: Faithful Latent Diffusion Model for Virtual Try-on
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（复旦大学脑智能科学与技术研究院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Suzhou Xiangji Technology Service Co., Ltd.（苏州湘记科技服务有限公司）
TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction using Diffusion Models
Machine Learning Group, CITEC Bielefeld University Germany（比勒菲尔德大学机器学习小组、CITEC）；Institute of Mathematics Technical University Berlin Germany（柏林技术大学数学研究所）
San Jose State University（圣何塞州立大学）；Carnegie Mellon University（卡内基梅隆大学）；Nanyang Technological University（南洋理工大学）
Virtual try-on has garnered interest as a neural rendering benchmark task to evaluate complex object transfer and scene composition. Recent works in virtual clothing try-on feature a plethora of possible architectural and data representation choices. However, they present little clarity on quantifying the isolated visual effect of each choice, nor do they specify the hyperparameter details that are key to experimental reproduction. Our work, ShineOn, approaches the try-on task from a bottom-up approach and aims to shine light on the visual and quantitative effects of each experiment. We build …
Once Is Enough: Lightweight DiT-Based Video Virtual Try-On via One-Time Garment Appearance Injection
School of computer science, Shanghai key laboratory of data science, Fudan University, China（计算机学院、数据科学重点实验室、复旦大学）；Tencent Youtu Lab, China（腾讯优图实验室）；Shanghai Ocean University, China（上海海洋大学）
ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On
South China University of Technology（华南理工大学）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；Guangdong Provincial Key Laboratory of Multimodal Big Data Intelligent Analysis（广东省多模态大数据智能分析重点实验室）
BooM-VVT: Boosting Mask-Free Video Virtual Try-On with Image-Level Pseudo Data
Nanjing University of Science and Technology（南京理工大学）；University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Meituan（美团）
