The 2D virtual try-on task has recently attracted a great interest from the research community, for its direct potential applications in online shopping as well as for its inherent and non-addressed scientific challenges. This task requires fitting an in-shop cloth image on the image of a person, which is highly challenging because it involves cloth warping, image compositing, and synthesizing. Casting virtual try-on into a supervised task faces a difficulty: available datasets are composed of pairs of pictures (cloth, person wearing the cloth). Thus, we have no access to ground-truth when the cloth on the person changes. State-of-the-art models solve this by masking the cloth information on the person with both a human parser and a pose estimator. Then, image synthesis modules are trained to reconstruct the person image from the masked person image and the cloth image. This procedure has several caveats: firstly, human parsers are prone to errors; secondly, it is a costly pre-processing step, which also has to be applied at inference time; finally, it makes the task harder than it is since the mask covers information that should be kept such as hands or accessories. In this paper, we propose a novel student-teacher paradigm where the teacher is trained in the standard way (reconstruction) before guiding the student to focus on the initial task (changing the cloth). The student additionally learns from an adversarial loss, which pushes it to follow the distribution of the real images. Consequently, the student exploits information that is masked to the teacher. A student trained without the adversarial loss would not use this information. Also, getting rid of both human parser and pose estimator at inference time allows obtaining a real-time virtual try-on.
FDM-MFVT: Few-step Sampling Diffusion Model for Mask-Free Virtual Try-On
School of Electronic Information Engineering, Beihang University, Beijing, China（北京航空航天大学电子信息工程学院）；Zhongguancun Academy, Beijing, China（中关村学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China（虚拟现实技术与系统国家重点实验室）
BooM-VVT: Boosting Mask-Free Video Virtual Try-On with Image-Level Pseudo Data
Nanjing University of Science and Technology（南京理工大学）；University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Meituan（美团）
Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image. Existing methods usually build up multi-stage frameworks to deal with clothes warping and body blending respectively, or rely heavily on intermediate parser-based labels which may be noisy or even inaccurate. To solve the above challenges, we propose a single-stage try-on framework by developing a novel Deformable Attention Flow (DAFlow), which applies the deformable attention scheme to multi-flow estimation. With pose keypoints as the guidance only, the self- and cross-defor…
The University of Hong Kong（香港大学）；Tencent AI Lab（腾讯AI实验室）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Tencent Data Platform（腾讯数据平台）
Image virtual try-on aims to fit a garment image (target clothes) to a person image. Prior methods are heavily based on human parsing. However, slightly-wrong segmentation results would lead to unrealistic try-on images with large artifacts. Inaccurate parsing misleads parser-based methods to produce visually unrealistic results where artifacts usually occur. A recent pioneering work employed knowledge distillation to reduce the dependency of human parsing, where the try-on images produced by a parser-based method are used as supervisions to train a "student" network without relying on segment…
Alibaba Group（阿里巴巴集团）；Zhejiang University（浙江大学）；Link2Do Technology Ltd（Link2Do科技有限公司）；Monash University（莫纳什大学）；University of California San Diego（加州大学圣地亚哥分校）
Virtual try-on(VTON) aims at fitting target clothes to reference person images, which is widely adopted in e-commerce.Existing VTON approaches can be narrowly categorized into Parser-Based(PB) and Parser-Free(PF) by whether relying on the parser information to mask the persons' clothes and synthesize try-on images. Although abandoning parser information has improved the applicability of PF methods, the ability of detail synthesizing has also been sacrificed. As a result, the distraction from original cloth may persistin synthesized images, especially in complicated postures and high resolution…
ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On
South China University of Technology（华南理工大学）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；Guangdong Provincial Key Laboratory of Multimodal Big Data Intelligent Analysis（广东省多模态大数据智能分析重点实验室）
National University of Singapore（新加坡国立大学）；Beijing Jiaotong University（北京交通大学）
Human parsing is attracting increasing research attention. In this work, we aim to push the frontier of human parsing by introducing the problem of multi-human parsing in the wild. Existing works on human parsing mainly tackle single-person scenarios, which deviates from real-world applications where multiple persons are present simultaneously with interaction and occlusion. To address the multi-human parsing problem, we introduce a new multi-human parsing (MHP) dataset and a novel multi-human parsing model named MH-Parser. The MHP dataset contains multiple persons captured in real-world scene…
University at Albany, State University of New York, USA（纽约州立大学奥尔巴尼分校）；Microsoft Corporation, Redmond, USA（微软公司雷德蒙德分部）；Samsung Research America AI Center, USA（三星美国研究公司人工智能中心）；University at Buffalo, State University of New York, USA（纽约州立大学布法罗分校）
We present Mask-guided Generative Adversarial Network (MagGAN) for high-resolution face attribute editing, in which semantic facial masks from a pre-trained face parser are used to guide the fine-grained image editing process. With the introduction of a mask-guided reconstruction loss, MagGAN learns to only edit the facial parts that are relevant to the desired attribute changes, while preserving the attribute-irrelevant regions (e.g., hat, scarf for modification `To Bald'). Further, a novel mask-guided conditioning strategy is introduced to incorporate the influence region of each attribute c…
DiffusionTrend: A Minimalist Approach to Virtual Fashion Try-On
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）；Skywork AI, Singapore（新加坡Skywork AI）
Previous virtual try-on methods usually focus on aligning a clothing item with a person, limiting their ability to exploit the complex pose, shape and skin color of the person, as well as the overall structure of the clothing, which is vital to photo-realistic virtual try-on. To address this potential weakness, we propose a fill in fabrics (FIFA) model, a self-supervised conditional generative adversarial network based framework comprised of a Fabricator and a unified virtual try-on pipeline with a Segmenter, Warper and Fuser. The Fabricator aims to reconstruct the clothing image when provided…
Face editing represents a popular research topic within the computer vision and image processing communities. While significant progress has been made recently in this area, existing solutions: (i) are still largely focused on low-resolution images, (ii) often generate editing results with visual artefacts, or (iii) lack fine-grained control and alter multiple (entangled) attributes at once, when trying to generate the desired facial semantics. In this paper, we aim to address these issues though a novel attribute editing approach called MaskFaceGAN that focuses on local attribute editing. The…
Sun Yat-sen University（中山大学）；South China Normal University（华南师范大学）
Virtual try-on system under arbitrary human poses has huge application potential, yet raises quite a lot of challenges, e.g. self-occlusions, heavy misalignment among diverse poses, and diverse clothes textures. Existing methods aim at fitting new clothes into a person can only transfer clothes on the fixed human pose, but still show unsatisfactory performances which often fail to preserve the identity, lose the texture details, and decrease the diversity of poses. In this paper, we make the first attempt towards multi-pose guided virtual try-on system, which enables transfer clothes on a pers…
Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared with real images, the desired performance cannot be achieved. To solve this problem, the previous method learned a model to improve the realism of the synthetic images. Different from the previous methods, this paper try to purify real image by extracting discriminative and robust features to convert outdoor real images to indoor synthetic images. In this…
Once Is Enough: Lightweight DiT-Based Video Virtual Try-On via One-Time Garment Appearance Injection
School of computer science, Shanghai key laboratory of data science, Fudan University, China（计算机学院、数据科学重点实验室、复旦大学）；Tencent Youtu Lab, China（腾讯优图实验室）；Shanghai Ocean University, China（上海海洋大学）
Do We Need to Design Specific Diffusion Models for Different Tasks? Try ONE-PIC
Nanjing University of Posts and Telecommunications（南京邮电大学）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, CAS（中国科学院自动化研究所）
DreamVTON: Customizing 3D Virtual Try-on with Personalized Diffusion Models
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；ByteDance（字节跳动）；Carnegie Mellon University（卡内基梅隆大学）；Research Institute of Multiple Agents and Embodied Intelligence（多智能体与具身智能研究院）；Peng Cheng Laboratory（鹏城实验室）
本文提出 DreamVTON 模型，通过结合个性化扩散模型、Densepose-guided ControlNet 和基于模板的优化机制，分别优化 3D 人体的几何与纹理，解决了个性化模块削弱多视图合成能力的问题，实现了高质量的定制化 3D 虚拟试穿。
Information Science and Technology College, Dalian Maritime University（大连海事大学信息科学技术学院）
Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared to real images, the desired performance cannot be achieved. Real images consist of multiple forms of light orientation, while synthetic images consist of a uniform light orientation. These features are considered to be characteristic of outdoor and indoor scenes, respectively. To solve this problem, the previous method learned a model to improve the reali…
PEMF-VTO: Point-Enhanced Video Virtual Try-on via Mask-free Paradigm
School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院，中国科学技术大学）；School of Electrical and Information Engineering, Tianjin University（电气与信息工程学院，天津大学）；Alibaba International Digital Commerce Group（阿里巴巴国际数字商业集团）
IIT Hyderabad（印度理工学院海得拉巴分校）；Media and Data Science Research Lab, Adobe（奥多比媒体与数据科学研究实验室）；Stanford University（斯坦福大学）；IIIT Hyderbad（印度国际信息技术学院海得拉巴分校）
Image-based virtual try-on for fashion has gained considerable attention recently. The task requires trying on a clothing item on a target model image. An efficient framework for this is composed of two stages: (1) warping (transforming) the try-on cloth to align with the pose and shape of the target model, and (2) a texture transfer module to seamlessly integrate the warped try-on cloth onto the target model image. Existing methods suffer from artifacts and distortions in their try-on output. In this work, we present SieveNet, a framework for robust image-based virtual try-on. Firstly, we int…
Single Stage Warped Cloth Learning and Semantic-Contextual Attention Feature Fusion for Virtual TryOn
Bharti School of Telecommunications, Indian Institute of Technology, Delhi, India（印度德里印度理工学院巴蒂电信学院）；Indian Institute of Information Technology, Sonepat, Haryana, India（印度哈里亚纳邦索内帕特印度信息技术学院）；Department of Electrical Engineering, Indian Institute of Technology, Delhi, India（印度德里印度理工学院电气工程系）
MOFA-VTON: More Fashion Possibilities with Fine-Grained Adaptations in Virtual Try-On
Harbin Institute of Technology（哈尔滨工业大学）；HiDream.ai Inc.（HiDream.ai公司）；Harbin Institute of Technology (Weihai) Qingdao Research Institute（哈尔滨工业大学（威海）青岛研究院）
