arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.02898cs.CVcs.MM
收藏

迈向统一的基于文本的行人检索:一个大规模多属性与语言搜索基准

Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark

Xi’an Jiaotong University(西安交通大学);Hefei University of Technology(合肥工业大学);Pengcheng Lab(鹏城实验室);National University of Singapore(新加坡国立大学)

Shuyu Yang, Yinan Zhou, Yaxiong Wang, Yujiao Wu, Li Zhu, Zhedong Zheng

AI总结:

该研究构建含151万图文对和27类属性的MALS大规模基准,并提出APTM联合预训练框架,在三个真实行人检索数据集上显著提升Recall@1性能。

摘要:

本文提出了一个用于基于文本的行人检索的大规模多属性与语言搜索数据集 MALS,并探索了在属性识别和图文匹配任务上同时进行预训练的可行性。具体而言,MALS 包含 1,510,330 个图文对,规模约为目前常用的 CUHK-PEDES 的 37.5 倍,并且所有图像均标注了 27 种属性。考虑到隐私问题和标注成本,我们利用现成的扩散模型生成该数据集。为验证从生成数据中学习的可行性,我们开发了一个新的联合属性提示学习与文本匹配学习(APTM)框架,以利用属性与文本之间的共享知识。顾名思义,APTM 包含一个属性提示学习流和一个文本匹配学习流。(1)属性提示学习利用属性提示实现图像—属性对齐,从而增强文本匹配学习。(2)文本匹配学习促进细粒度细节上的表示学习,进而推动属性提示学习。大量实验验证了在 MALS 上进行预训练的有效性,APTM 在三个具有挑战性的真实世界基准上取得了最先进的检索性能。特别是,APTM 在 CUHK-PEDES、ICFG-PEDES 和 RSTPReid 数据集上的 Recall@1 准确率分别稳定提升了 +6.96%、+7.68% 和 +16.95%,优势明显。

英文摘要:

In this paper, we introduce a large Multi-Attribute and Language Search dataset for text-based person retrieval, called MALS, and explore the feasibility of performing pre-training on both attribute recognition and image-text matching tasks in one stone. In particular, MALS contains 1,510,330 image-text pairs, which is about 37.5 times larger than prevailing CUHK-PEDES, and all images are annotated with 27 attributes. Considering the privacy concerns and annotation costs, we leverage the off-the-shelf diffusion models to generate the dataset. To verify the feasibility of learning from the generated data, we develop a new joint Attribute Prompt Learning and Text Matching Learning (APTM) framework, considering the shared knowledge between attribute and text. As the name implies, APTM contains an attribute prompt learning stream and a text matching learning stream. (1) The attribute prompt learning leverages the attribute prompts for image-attribute alignment, which enhances the text matching learning. (2) The text matching learning facilitates the representation learning on fine-grained details, and in turn, boosts the attribute prompt learning. Extensive experiments validate the effectiveness of the pre-training on MALS, achieving state-of-the-art retrieval performance via APTM on three challenging real-world benchmarks. In particular, APTM achieves a consistent improvement of +6.96%, +7.68%, and +16.95% Recall@1 accuracy on CUHK-PEDES, ICFG-PEDES, and RSTPReid datasets by a clear margin, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑