Contrastive Language-Image Pre-Training (CLIP) has shown impressive performance in short-term Person Re-Identification (ReID) due to its ability to extract high-level semantic features of pedestrians, yet its direct application to Cloth-Changing Person Re-Identification (CC-ReID) faces challenges due to CLIP's image encoder overly focusing on clothes clues. To address this, we propose a novel framework called CLIP-Driven Cloth-Agnostic Feature Learning (CCAF) for CC-ReID. Accordingly, two modules were custom-designed: the Invariant Feature Prompting (IFP) and the Clothes Feature Minimization (CFM). These modules guide the model to extract cloth-agnostic features positively and attenuate clothes-related features negatively. Specifically, IFP is designed to extract fine-grained semantic features unrelated to clothes from the raw image, guided by the cloth-agnostic text prompts. This module first covers the clothes in the raw image at the pixel level to obtain the shielding image and then utilizes CLIP's knowledge to generate cloth-agnostic text prompts. Subsequently, it aligns the raw image-text and the raw image-shielding image in the feature space, emphasizing discriminative clues related to identity but unrelated to clothes. Furthermore, CFM is designed to examine and weaken the image encoder's ability to extract clothes features. It first generates text prompts corresponding to clothes pixels. Then, guided by these clothes text prompts, it iteratively examines and disentangles clothes features from pedestrian features, ultimately retaining inherent discriminative features. Extensive experiments have demonstrated the effectiveness of the proposed CCAF, achieving new state-of-the-art performance on several popular CC-ReID benchmarks without any additional inference time.
Shenzhen Institutes of Advanced Technology, Chinese Academy of Science（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Microelectronics of the Chinese Academy of Sciences（中国科学院微电子研究所）
Recent years have witnessed great progress in person re-identification (re-id). Several academic benchmarks such as Market1501, CUHK03 and DukeMTMC play important roles to promote the re-id research. To our best knowledge, all the existing benchmarks assume the same person will have the same clothes. While in real-world scenarios, it is very often for a person to change clothes. To address the clothes changing person re-id problem, we construct a novel large-scale re-id benchmark named ClOthes ChAnging Person Set (COCAS), which provides multiple images of the same identity with different cloth…
Sun Yat-sen University（中山大学）；Peng Cheng Laboratory（鹏城实验室）
Person re-identification (re-id), the process of matching pedestrian images across different camera views, is an important task in visual surveillance. Substantial development of re-id has recently been observed, and the majority of existing models are largely dependent on color appearance and assume that pedestrians do not change their clothes across camera views. This limitation, however, can be an issue for re-id when tracking a person at different places and at different time if that person (e.g., a criminal suspect) changes his/her clothes, causing most existing methods to fail, since the…
Tianjin University（天津大学）；Hefei University of Technology（合肥工业大学）
Person reidentification (ReID) is a very hot research topic in machine learning and computer vision, and many person ReID approaches have been proposed; however, most of these methods assume that the same person has the same clothes within a short time interval, and thus their visual appearance must be similar. However, in an actual surveillance environment, a given person has a great probability of changing clothes after a long time span, and they also often take different personal belongings with them. When the existing person ReID methods are applied in this type of case, almost all of them…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
The key to address clothes-changing person re-identification (re-id) is to extract clothes-irrelevant features, e.g., face, hairstyle, body shape, and gait. Most current works mainly focus on modeling body shape from multi-modality information (e.g., silhouettes and sketches), but do not make full use of the clothes-irrelevant information in the original RGB images. In this paper, we propose a Clothes-based Adversarial Loss (CAL) to mine clothes-irrelevant features from the original RGB images by penalizing the predictive power of re-id model w.r.t. clothes. Extensive experiments demonstrate t…
Person re-identification (Re-ID) aims to match a target person across camera views at different locations and times. Existing Re-ID studies focus on the short-term cloth-consistent setting, under which a person re-appears in different camera views with the same outfit. A discriminative feature representation learned by existing deep Re-ID models is thus dominated by the visual appearance of clothing. In this work, we focus on a much more difficult yet practical setting where person matching is conducted over long-duration, e.g., over days and months and therefore inevitably under the new chall…
Person re-identification (ReID) is now an active research topic for AI-based video surveillance applications such as specific person search, but the practical issue that the target person(s) may change clothes (clothes inconsistency problem) has been overlooked for long. For the first time, this paper systematically studies this problem. We first overcome the difficulty of lack of suitable dataset, by collecting a small yet representative real dataset for testing whilst building a large realistic synthetic dataset for training and deeper studies. Facilitated by our new datasets, we are able to…
Person re-identification (re-ID) aims to recognize instances of the same person contained in multiple images taken across different cameras. Existing methods for re-ID tend to rely heavily on the assumption that both query and gallery images of the same person have the same clothing. Unfortunately, this assumption may not hold for datasets captured over long periods of time (e.g., weeks, months or years). To tackle the re-ID problem in the context of clothing changes, we propose a novel representation learning model which is able to generate a body shape feature representation without being af…
With the rise of deep learning methods, person Re-Identification (ReID) performance has been improved tremendously in many public datasets. However, most public ReID datasets are collected in a short time window in which persons' appearance rarely changes. In real-world applications such as in a shopping mall, the same person's clothing may change, and different persons may wearing similar clothes. All these cases can result in an inconsistent ReID performance, revealing a critical problem that current ReID models heavily rely on person's apparels. Therefore, it is critical to learn an apparel…
Content and Salient Semantics Collaboration for Cloth-Changing Person Re-Identification
School of Computer Science, Fudan University（复旦大学计算机科学学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Data Science, Fudan University（复旦大学数据科学学院）
SCORE: SubDistribution-aware Collaborative Knowledge Reinforcing for Cloth-Hybrid Lifelong Person Re-Identification
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Intelligent Science & Technology Academy of CASIC（中国航天科工集团智能科技研究院）
Unsupervised Long-Term Person Re-Identification with Clothes Change
China Academy of Aerospace Science and Innovation（中国航天科技创新研究院）；Tsinghua University（清华大学）；University of Surrey（萨里大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Tianjin University（天津大学）；University of South Carolina Columbia（南卡罗来纳大学哥伦比亚分校）
Person re-identification (Re-ID) is a classical computer vision task and has achieved great progress so far. Recently, long-term Re-ID with clothes-changing has attracted increasing attention. However, existing methods mainly focus on image-based setting, where richer temporal information is overlooked. In this paper, we focus on the relatively new yet practical problem of clothes-changing video-based person re-identification (CCVReID), which is less studied. We systematically study this problem by simultaneously considering the challenge of the clothes inconsistency issue and the temporal inf…
Causal Clothes-Invariant Feature Learning for Cloth-Changing Person Re-ID
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络安全学院）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Data Science, University of Science and Technology of China（中国科学技术大学数据科学学院）；Wuhan University（武汉大学）
Cloth-changing person re-identification (CC-ReID), which aims to match person identities under clothing changes, is a new rising research topic in recent years. However, typical biometrics-based CC-ReID methods often require cumbersome pose or body part estimators to learn cloth-irrelevant features from human biometric traits, which comes with high computational costs. Besides, the performance is significantly limited due to the resolution degradation of surveillance images. To address the above limitations, we propose an effective Identity-Sensitive Knowledge Propagation framework (DeSKPro) f…
University of Oxford（牛津大学）；University of Surrey（萨里大学）
Existing person re-identification (re-id) works mostly consider short-term application scenarios without clothes change. In real-world, however, we often dress differently across space and time. To solve this contrast, a few recent attempts have been made on long-term re-id with clothes change. Currently, one of the most significant limitations in this field is the lack of a large realistic benchmark. In this work, we contribute a large, realistic long-term person re-identification benchmark, named as DeepChange. It has several unique characteristics: (1) Realistic and rich personal appearance…
Identity-Guided Collaborative Learning for Cloth-Changing Person Reidentification
Shandong Artificial Intelligence Institute（山东人工智能研究院）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））；Tianjin University of Technology（天津理工大学）；Monash University（莫纳什大学）；Tongji University（同济大学）；Hefei University of Technology（合肥工业大学）
University of Science and Technology of China（中国科学技术大学）；Alibaba Cloud Computing Ltd.（阿里云计算有限公司）；University of Michigan（密歇根大学）
Cloth-Changing person re-identification (CC-ReID) aims at matching the same person across different locations over a long-duration, e.g., over days, and therefore inevitably meets challenge of changing clothing. In this paper, we focus on handling well the CC-ReID problem under a more challenging setting, i.e., just from a single image, which enables high-efficiency and latency-free pedestrian identify for real-time surveillance applications. Specifically, we introduce Gait recognition as an auxiliary task to drive the Image ReID model to learn cloth-agnostic representations by leveraging pers…
A Semantic-aware Attention and Visual Shielding Network for Cloth-changing Person Re-identification
Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin University of Technology（天津理工大学计算机视觉与系统教育部重点实验室）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）
Discriminative Pedestrian Features and Gated Channel Attention for Clothes-Changing Person Re-Identification
College of Computer Science；Technology Nanjing University of Aeronautics；School of Information；Artificial Intelligence Anhui Agriculture University Hefei, China
Inception Institute of Artificial Intelligence (IIAI)（Inception人工智能研究院（IIAI））；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Recently, large-scale synthetic datasets are shown to be very useful for generalizable person re-identification. However, synthesized persons in existing datasets are mostly cartoon-like and in random dress collocation, which limits their performance. To address this, in this work, an automatic approach is proposed to directly clone the whole outfits from real-world person images to virtual 3D characters, such that any virtual person thus created will appear very similar to its real-world counterpart. Specifically, based on UV texture mapping, two cloning methods are designed, namely registere…
OC4-ReID: Occluded Cloth-Changing Person Re-Identification
Beijing University of Posts and Telecommunications（北京邮电大学）；South China University of Technology（华南理工大学）；Nanjing University of Information Science & Technology（南京信息科学技术大学）；University of Oxford（牛津大学）；Nanjing University（南京大学）
Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification
Shanghai Key Lab of Intelligent Information Processing, Fudan University（复旦大学上海市智能信息处理重点实验室）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；Fudan ISTBI—ZJNU Algorithm Centre for Brain-inspired Intelligence, Fudan University（复旦大学ISTBI-浙江师范大学类脑智能算法中心）
PGDS: Pose-Guidance Deep Supervision for Mitigating Clothes-Changing in Person Re-Identification
University of Science, Vietnam National University, Ho Chi Minh City（越南国家大学胡志明市校区科学大学）；John von Neumann Institute, Vietnam National University, Ho Chi Minh City（越南国家大学胡志明市校区约翰·冯·诺依曼研究所）；University of Arkansas（阿肯色大学）；Northwestern University（西北大学）
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；Tokyo University（东京大学）；Fudan University（复旦大学）；Tencent（腾讯）
Cloth changing person re-identification(Re-ID) can work under more complicated scenarios with higher security than normal Re-ID and biometric techniques and is therefore extremely valuable in applications. Meanwhile, higher flexibility in appearance always leads to more similar-looking confusing images, which is the weakness of the widely used retrieval methods. In this work, we shed light on how to handle these similar images. Specifically, we propose a novel retrieval-verification framework. Given an image, the retrieval module can search for similar images quickly. Our proposed verification…
Image-Text-Image Knowledge Transfer for Lifelong Person Re-Identification with Hybrid Clothing States
School of Automation, Northwestern Polytechnical University（自动化学院，西北工业大学）；School of Computer Science, Shanghai Key Lab of Intelligent Information Processing, Fudan University（计算机学院，上海智能信息处理重点实验室，复旦大学）；Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院）
Cloth-changing person re-identification (re-ID) is a new rising research topic that aims at retrieving pedestrians whose clothes are changed. This task is quite challenging and has not been fully studied to date. Current works mainly focus on body shape or contour sketch, but they are not robust enough due to view and posture variations. The key to this task is to exploit cloth-irrelevant cues. This paper proposes a semantic-guided pixel sampling approach for the cloth-changing person re-ID task. We do not explicitly define which feature to extract but force the model to automatically learn cl…
See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification
Sanya Science and Education Innovation Park, Wuhan University of Technology（武汉理工大学三亚科学教育创新园）；Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学交通运输物联网重点实验室）；Hubei Key Laboratory of Big Data Intelligent Analysis and Application, School of Computer Science and Information Engineering, Hubei University（湖北大数据智能分析与应用重点实验室）；National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（国家多媒体软件工程研究中心）；School of Computing, Macquarie University（麦考瑞大学计算机学院）；Rapid-Rich Object Search Lab, School of Electrical and Electronic Engineering, Nanyang Technological University（快速富对象搜索实验室）
Person search in media has seen increasing potential in Internet applications, such as video clipping and character collection. This task is common but overlooked by previous person search works which focus on surveillance scenes. The media scenarios have some different challenges from surveillance scenes. For example, a person may change his clothes frequently. To alleviate this issue, this paper proposes a Unified Detector and Graph Network (UDGNet) for person search in media. UDGNet is the first person search framework to detect and re-identify the human body and head simultaneously. Specif…
Try Harder: Hard Sample Generation and Learning for Clothes-Changing Person Re-ID
School of Computer Science and Engineering, Beihang University（北航计算机科学与工程学院）；School of Artificial Intelligence, Beihang University（北航人工智能学院）；Zhongguancun Academy（中关村学院）
Person Re-Identification (ReID) aims to recognize a person-of-interest across different places and times. Existing ReID methods rely on images or videos collected using RGB cameras. They extract appearance features like clothes, shoes, hair, etc. Such features, however, can change drastically from one day to the next, leading to inability to identify people over extended time periods. In this paper, we introduce RF-ReID, a novel approach that harnesses radio frequency (RF) signals for longterm person ReID. RF signals traverse clothes and reflect off the human body; thus they can be used to ext…
We address the challenging task of video-based person re-identification. Recent works have shown that splitting the video sequences into clips and then aggregating clip based similarity is appropriate for the task. We show that using a learned clip similarity aggregation function allows filtering out hard clip pairs, e.g. where the person is not clearly visible, is in a challenging pose, or where the poses in the two clips are too different to be informative. This allows the method to focus on clip-pairs which are more informative for the task. We also introduce the use of 3D CNNs for video-ba…
In video-surveillance, person re-identification is the task of recognising whether an individual has already been observed over a network of cameras. Typically, this is achieved by exploiting the clothing appearance, as classical biometric traits like the face are impractical in real-world video surveillance scenarios. Clothing appearance is represented by means of low-level \textit{local} and/or \textit{global} features of the image, usually extracted according to some part-based body model to treat different body parts (e.g. torso and legs) independently. This paper provides a comprehensive …
Harbin Engineering University（哈尔滨工程大学）；Harbin Institute of Technology（哈尔滨工业大学）；National University of Singapore（新加坡国立大学）
Pose variation is one of the key factors which prevents the network from learning a robust person re-identification (Re-ID) model. To address this issue, we propose a novel person pose-guided image generation method, which is called the semantic attention network. The network consists of several semantic attention blocks, where each block attends to preserve and update the pose code and the clothing textures. The introduction of the binary segmentation mask and the semantic parsing is important for seamlessly stitching foreground and background in the pose-guided image generation. Compared wit…
University of Chinese Academy of Sciences（中国科学院大学）；AI Research of JD.com（京东人工智能研究院）
Person re-identification (Re-ID) aims at retrieving an input person image from a set of images captured by multiple cameras. Although recent Re-ID methods have made great success, most of them extract features in terms of the attributes of clothing (e.g., color, texture). However, it is common for people to wear black clothes or be captured by surveillance systems in low light illumination, in which cases the attributes of the clothing are severely missing. We call this problem the Black Re-ID problem. To solve this problem, rather than relying on the clothing information, we propose to exploi…
QA-ReID: Quality-Aware Query-Adaptive Convolution Leveraging Fused Global and Structural Cues for Clothes-Changing ReID
School of Business, Stevens Institute of Technology（斯蒂文斯理工学院商学院）；Electrical and Computer Engineering, University of California, San Diego（加州大学圣地亚哥分校电子与计算机工程系）；Tandon School of Engineering, New York University（纽约大学塔恩工学院）；Institute of Operations Research and Analytics, National University of Singapore（新加坡国立大学运筹与分析研究所）
