梯度注意力引导的双掩码协同框架用于鲁棒的基于文本的行人检索
Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval
AI总结：
本研究提出GA-DMS框架，结合WebPerson大规模数据集，通过梯度注意力引导的双掩码机制和掩码标记预测，改进CLIP在行人检索中的细粒度跨模态对齐，实现多基准最优性能。
摘要：
尽管对比语言-图像预训练（CLIP）在多种视觉任务中表现出强大的性能，但其在行人表示学习中的应用面临两个关键挑战：（i）缺乏以行人图像为中心的大规模标注视觉-语言数据，以及（ii）全局对比学习的固有局限性，这种学习难以保持对细粒度匹配至关重要的判别性局部特征，同时容易受到噪声文本标记的影响。本研究通过在数据整理和模型架构方面的协同改进，推进了CLIP在行人表示学习中的应用。首先，我们开发了一种抗噪声的数据构建流程，利用多模态大语言模型（MLLMs）的上下文学习能力自动过滤和标注网络来源的图像。这产生了WebPerson，一个包含500万高质量行人中心图像-文本对的大规模数据集。其次，我们引入了GA-DMS（梯度注意力引导的双掩码协同）框架，该框架通过基于梯度-注意力相似度得分自适应地掩码噪声文本标记来改进跨模态对齐。此外，我们加入了掩码标记预测目标，迫使模型预测信息丰富的文本标记，增强细粒度语义表示学习。大量实验表明，GA-DMS在多个基准上实现了最先进的性能。
英文摘要：
Although Contrastive Language-Image Pre-training (CLIP) exhibits strong performance across diverse vision tasks, its application to person representation learning faces two critical challenges: (i) the scarcity of large-scale annotated vision-language data focused on person-centric images, and (ii) the inherent limitations of global contrastive learning, which struggles to maintain discriminative local features crucial for fine-grained matching while remaining vulnerable to noisy text tokens. This work advances CLIP for person representation learning through synergistic improvements in data curation and model architecture. First, we develop a noise-resistant data construction pipeline that leverages the in-context learning capabilities of MLLMs to automatically filter and caption web-sourced images. This yields WebPerson, a large-scale dataset of 5M high-quality person-centric image-text pairs. Second, we introduce the GA-DMS (Gradient-Attention Guided Dual-Masking Synergetic) framework, which improves cross-modal alignment by adaptively masking noisy textual tokens based on the gradient-attention similarity score. Additionally, we incorporate masked token prediction objectives that compel the model to predict informative text tokens, enhancing fine-grained semantic representation learning. Extensive experiments show that GA-DMS achieves state-of-the-art performance across multiple benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
