arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.06385cs.CV
收藏

AMNS:面向文本到图像行人检索的注意力加权选择性掩码与噪声标签抑制

AMNS: Attention-Weighted Selective Mask and Noise Label Suppression for Text-to-Image Person Retrieval

Shenzhen Institute for Advanced Study(深圳高等研究院);University of Electronic Science and Technology of China(电子科技大学)

Runqing Zhang, Xue Zhou

AI总结:

本文针对文本到图像行人检索中的噪声对应和随机掩码破坏语义问题,提出噪声标签抑制与注意力加权选择性掩码方法,通过BSDM和WAF损失及EMA编码器提升检索性能。

摘要:

现有的大多数文本到图像行人检索方法通常假设训练图像-文本对是完全对齐的;然而,由于图像质量差和标注错误,存在噪声对应(NC)问题(即不正确或不可靠的对齐)。此外,随机掩码增强可能会无意中丢弃关键的语义内容,从而在图像和文本描述之间引入噪声匹配。为了解决上述两个挑战,我们提出了一种噪声标签抑制方法来缓解NC问题,并提出了一种注意力加权选择性掩码(AWM)策略来解决随机掩码引起的问题。具体而言,双向相似性分布匹配(BSDM)损失使模型能够有效地从正样本对中学习,同时防止模型过度依赖它们,从而降低对噪声标签过拟合的风险。与此相结合,权重调整焦点(WAF)损失提高了模型处理困难样本的能力。此外,AWM通过图像编码器的EMA版本处理原始图像,选择性地保留与文本具有强语义连接的token,从而实现更好的特征提取。大量实验证明了我们的方法在解决噪声相关问题方面的有效性,并提升了检索性能。

英文摘要:

Most existing text-to-image person retrieval methods usually assume that the training image-text pairs are perfectly aligned; however, the noisy correspondence(NC) issue (i.e., incorrect or unreliable alignment) exists due to poor image quality and labeling errors. Additionally, random masking augmentation may inadvertently discard critical semantic content, introducing noisy matches between images and text descriptions. To address the above two challenges, we propose a noise label suppression method to mitigate NC and an Attention-Weighted Selective Mask (AWM) strategy to resolve the issues caused by random masking. Specifically, the Bidirectional Similarity Distribution Matching (BSDM) loss enables the model to effectively learn from positive pairs while preventing it from over-relying on them, thereby mitigating the risk of overfitting to noisy labels. In conjunction with this, Weight Adjustment Focal (WAF) loss improves the model's ability to handle hard samples. Furthermore, AWM processes raw images through an EMA version of the image encoder, selectively retaining tokens with strong semantic connections to the text, enabling better feature extraction. Extensive experiments demonstrate the effectiveness of our approach in addressing noise-related issues and improving retrieval performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑