arXiv 2409.195632024-09-29cs.CVcs.AI

基于CLIP的相机无关特征学习用于相机内行人重识别

CLIP-based Camera-Agnostic Feature Learning for Intra-camera Person Re-Identification

Tangshan Research Institute, Southwest Jiaotong University（西南交通大学唐山研究院）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, China（教育部可持续城市智能交通工程研究中心）；Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province（四川省产业链协同与信息支撑技术重点实验室）

Xuan Tan, Xun Gong, Yang Xiang

备注：: Submitted to IEEE TCSVT

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对相机内监督行人重识别中跨相机标注缺失问题，提出基于CLIP的CCAFL框架，通过相机内判别学习和相机间对抗学习学习相机无关特征，在MSMT17上mAP达58.9%，超越现有方法7.6%。

摘要：

对比语言-图像预训练（CLIP）模型在传统行人重识别（ReID）任务中表现出色，因为其具有为行人图像生成文本描述的内在优势。然而，将CLIP直接应用于相机内监督行人重识别（ICS ReID）面临挑战。ICS ReID要求在每个相机内独立进行身份标注，相机之间不建立关联，这限制了基于文本增强方法的有效性。为解决这一问题，我们提出了一种新的框架，称为基于CLIP的相机无关特征学习（CCAFL），用于ICS ReID。相应地，设计了两个定制模块来引导模型主动学习相机无关的行人特征：相机内判别学习（ICDL）和相机间对抗学习（ICAL）。具体来说，我们首先为相机内行人图像建立可学习的文本提示，以获得用于后续相机内和相机间学习的关键语义监督信号。然后，我们设计ICDL，通过考虑每个相机内的困难正样本和困难负样本来增大类间差异，从而学习相机内更细粒度的行人特征。此外，我们提出ICAL，通过惩罚模型预测行人图像来源相机的能力来减少相机间行人特征差异，从而增强模型从不同视角识别行人的能力。在流行的ReID数据集上的大量实验证明了我们方法的有效性。特别是在具有挑战性的MSMT17数据集上，我们达到了58.9%的mAP精度，超过最先进方法7.6%。代码将公开于：https://github.com/Trangle12/CCAFL。

英文摘要：

Contrastive Language-Image Pre-Training (CLIP) model excels in traditional person re-identification (ReID) tasks due to its inherent advantage in generating textual descriptions for pedestrian images. However, applying CLIP directly to intra-camera supervised person re-identification (ICS ReID) presents challenges. ICS ReID requires independent identity labeling within each camera, without associations across cameras. This limits the effectiveness of text-based enhancements. To address this, we propose a novel framework called CLIP-based Camera-Agnostic Feature Learning (CCAFL) for ICS ReID. Accordingly, two custom modules are designed to guide the model to actively learn camera-agnostic pedestrian features: Intra-Camera Discriminative Learning (ICDL) and Inter-Camera Adversarial Learning (ICAL). Specifically, we first establish learnable textual prompts for intra-camera pedestrian images to obtain crucial semantic supervision signals for subsequent intra- and inter-camera learning. Then, we design ICDL to increase inter-class variation by considering the hard positive and hard negative samples within each camera, thereby learning intra-camera finer-grained pedestrian features. Additionally, we propose ICAL to reduce inter-camera pedestrian feature discrepancies by penalizing the model's ability to predict the camera from which a pedestrian image originates, thus enhancing the model's capability to recognize pedestrians from different viewpoints. Extensive experiments on popular ReID datasets demonstrate the effectiveness of our approach. Especially, on the challenging MSMT17 dataset, we arrive at 58.9\% in terms of mAP accuracy, surpassing state-of-the-art methods by 7.6\%. Code will be available at: https://github.com/Trangle12/CCAFL.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。