基于原型对比学习的CLIP微调用于物体重识别
Prototypical Contrastive Learning-based CLIP Fine-tuning for Object Re-identification
AI总结：
本文提出基于原型对比学习的CLIP微调方法，用于提升物体重识别性能，无需提示学习并实现优于CLIP-ReID的性能。
摘要：
本文旨在适应大规模预训练视觉-语言模型，如对比语言-图像预训练（CLIP），以提升跨多种监督设置的物体重识别（Re-ID）性能。尽管提示学习已使最近的工作CLIP-ReID取得有希望的性能，但由于重识别任务中缺乏语义标签，提示学习的内在机制和必要性仍不清楚。在本文中，我们首先分析提示学习在CLIP-ReID中的作用并识别其局限性。基于我们的研究，我们提出了一种简单而有效的方法来适应CLIP用于监督物体Re-ID。我们的方法直接通过原型对比学习（PCL）损失微调CLIP的图像编码器，从而消除了对提示学习的需要。在人和车辆Re-ID数据集上的实验结果表明，我们的方法在与CLIP-ReID相比具有竞争力。此外，我们将基于PCL的CLIP微调方法扩展到无监督场景，在其中我们实现了最先进的性能。代码可在https://github.com/RikoLi/PCL-CLIP上获得。
英文摘要：
This work aims to adapt large-scale pre-trained vision-language models, such as contrastive language-image pretraining (CLIP), to enhance the performance of object reidentification (Re-ID) across various supervision settings. Although prompt learning has enabled a recent work named CLIP-ReID to achieve promising performance, the underlying mechanisms and the necessity of prompt learning remain unclear due to the absence of semantic labels in ReID tasks. In this work, we first analyze the role prompt learning in CLIP-ReID and identify its limitations. Based on our investigations, we propose a simple yet effective approach to adapt CLIP for supervised object Re-ID. Our approach directly fine-tunes the image encoder of CLIP using a prototypical contrastive learning (PCL) loss, eliminating the need for prompt learning. Experimental results on both person and vehicle Re-ID datasets demonstrate the competitiveness of our method compared to CLIP-ReID. Furthermore, we extend our PCL-based CLIP fine-tuning approach to unsupervised scenarios, where we achieve state-of-the art performance. Code is available at https://github.com/RikoLi/PCL-CLIP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
