面向文本到图像行人检索的跨模态自适应双重关联
Cross-Modal Adaptive Dual Association for Text-to-Image Person Retrieval
AI总结：
本文针对文本到图像行人检索中双向关联不一致问题，提出跨模态自适应双重关联方法CADA，通过文本Token到图像Patch关联和图像区域到文本属性关联实现精细双向匹配，实验验证了其优越性。
摘要：
文本到图像行人重识别（ReID）旨在根据给定的文本描述检索行人图像。其关键挑战在于学习视觉与文本模态中详细信息之间的关系。现有工作主要聚焦于学习一个潜在空间以缩小模态差距，并进一步构建两种模态之间的局部对应关系。然而，这些方法假设图像到文本和文本到图像的关联是与模态无关的，从而导致关联结果不够理想。本文揭示了图像到文本关联与文本到图像关联之间的差异，并提出CADA：跨模态自适应双重关联，精细地构建双向的图像-文本详细关联。我们的方法采用基于解码器的自适应双重关联模块，使视觉与文本模态之间能够充分交互，从而实现双向且自适应的跨模态对应关联。具体而言，本文提出了一种双向关联机制：文本Token到图像Patch的关联（ATP）以及图像区域到文本属性的关联（ARA）。我们基于错误关联会导致跨模态特征聚合时发生特征失真这一事实，对ATP进行自适应建模。对于ARA的建模，由于属性通常是行人最首要的区分线索，我们提出通过利用相关图像区域预测被掩码的文本短语来探索属性级关联。最后，我们学习文本与图像之间的双重关联，实验结果证明了该双重关联框架的优越性。代码将公开发布。
英文摘要：
Text-to-image person re-identification (ReID) aims to retrieve images of a person based on a given textual description. The key challenge is to learn the relations between detailed information from visual and textual modalities. Existing works focus on learning a latent space to narrow the modality gap and further build local correspondences between two modalities. However, these methods assume that image-to-text and text-to-image associations are modality-agnostic, resulting in suboptimal associations. In this work, we show the discrepancy between image-to-text association and text-to-image association and propose CADA: Cross-Modal Adaptive Dual Association that finely builds bidirectional image-text detailed associations. Our approach features a decoder-based adaptive dual association module that enables full interaction between visual and textual modalities, allowing for bidirectional and adaptive cross-modal correspondence associations. Specifically, the paper proposes a bidirectional association mechanism: Association of text Tokens to image Patches (ATP) and Association of image Regions to text Attributes (ARA). We adaptively model the ATP based on the fact that aggregating cross-modal features based on mistaken associations will lead to feature distortion. For modeling the ARA, since the attributes are typically the first distinguishing cues of a person, we propose to explore the attribute-level association by predicting the masked text phrase using the related image region. Finally, we learn the dual associations between texts and images, and the experimental results demonstrate the superiority of our dual formulation. Codes will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
