CPCL：面向弱监督文本行人检索的跨模态原型对比学习
CPCL: Cross-Modal Prototypical Contrastive Learning for Weakly Supervised Text-based Person Retrieval
AI总结：
本文提出CPCL方法，通过引入CLIP模型、原型多模态记忆模块和离群伪标签挖掘模块，利用跨模态原型对比学习解决弱监督文本行人检索中的类内差异问题，并在多个基准上验证了其有效性和泛化性。
摘要：
弱监督文本行人检索旨在利用文本描述检索目标行人图像，不依赖身份标注，因而更具挑战性和实用性。其主要难点在于类内差异，包括模态内特征变化和跨模态语义鸿沟。以往工作聚焦于实例级样本，忽略了每个行人固有且不变的原型特征。为此，我们提出了一种跨模态原型对比学习（CPCL）方法。具体而言，CPCL将CLIP模型引入弱监督文本行人检索，将视觉和文本实例映射到共享潜在空间。随后，所提出的原型多模态记忆（PMM）模块通过混合跨模态匹配（HCM）模块以多对多映射方式捕获属于同一行人的图像-文本对之间异构模态的关联。此外，离群伪标签挖掘（OPLM）模块进一步从各模态中区分有价值的离群样本，通过挖掘图像-文本对之间的隐式关系来构建更可靠的聚类。我们在弱监督文本行人检索的常用基准上进行了大量实验，验证了CPCL的有效性和泛化性。
英文摘要：
Weakly supervised text-based person retrieval seeks to retrieve images of a target person using textual descriptions, without relying on identity annotations and is more challenging and practical. The primary challenge is the intra-class differences, encompassing intra-modal feature variations and cross-modal semantic gaps. Prior works have focused on instance-level samples and ignored prototypical features of each person which are intrinsic and invariant. Toward this, we propose a Cross-Modal Prototypical Contrastive Learning (CPCL) method. In practice, the CPCL introduces the CLIP model to weakly supervised text-based person retrieval to map visual and textual instances into a shared latent space. Subsequently, the proposed Prototypical Multi-modal Memory (PMM) module captures associations between heterogeneous modalities of image-text pairs belonging to the same person through the Hybrid Cross-modal Matching (HCM) module in a many-to-many mapping fashion. Moreover, the Outlier Pseudo Label Mining (OPLM) module further distinguishes valuable outlier samples from each modality, enhancing the creation of more reliable clusters by mining implicit relationships between image-text pairs. We conduct extensive experiments on popular benchmarks of weakly supervised text-based person retrieval, which validate the effectiveness, generalizability of CPCL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
