面向可泛化行人重识别的鲁棒视觉-语义嵌入学习
Learning Robust Visual-Semantic Embedding for Generalizable Person Re-identification
AI总结：
本文提出多模态等效Transformer（MMET）和掩码多模态建模策略（MMM），通过联合学习视觉与语义特征并动态掩码图像块和文本标记，提升可泛化行人重识别的跨域泛化能力。
摘要：
可泛化行人重识别（Re-ID）是机器学习和计算机视觉中一个非常热门的研究课题，由于其在公共安全和视频监控中的多种应用，在现实场景中发挥着重要作用。然而，以往的方法主要关注视觉表示学习，而忽视了在训练过程中探索语义特征的潜力，这容易导致在适应新领域时泛化能力较差。在本文中，我们提出了一种称为MMET的多模态等效Transformer，用于分别在视觉、文本和视觉-文本任务上进行更鲁棒的视觉-语义嵌入学习。为了进一步增强Transformer背景下鲁棒特征学习，我们引入了一种称为掩码多模态建模策略（MMM）的动态掩码机制，用于同时掩码图像块和文本标记，该机制可以联合作用于多模态或单模态数据，并显著提升可泛化行人Re-ID的性能。在基准数据集上的大量实验表明，我们的方法相比以往方法具有竞争力的性能。我们希望该方法能够推动视觉-语义表示学习的研究。我们的源代码也已在https://github.com/JeremyXSC/MMET公开提供。
英文摘要：
Generalizable person re-identification (Re-ID) is a very hot research topic in machine learning and computer vision, which plays a significant role in realistic scenarios due to its various applications in public security and video surveillance. However, previous methods mainly focus on the visual representation learning, while neglect to explore the potential of semantic features during training, which easily leads to poor generalization capability when adapted to the new domain. In this paper, we propose a Multi-Modal Equivalent Transformer called MMET for more robust visual-semantic embedding learning on visual, textual and visual-textual tasks respectively. To further enhance the robust feature learning in the context of transformer, a dynamic masking mechanism called Masked Multimodal Modeling strategy (MMM) is introduced to mask both the image patches and the text tokens, which can jointly works on multimodal or unimodal data and significantly boost the performance of generalizable person Re-ID. Extensive experiments on benchmark datasets demonstrate the competitive performance of our method over previous approaches. We hope this method could advance the research towards visual-semantic representation learning. Our source code is also publicly available at https://github.com/JeremyXSC/MMET.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
