arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.04496cs.CV
收藏

PersonMAE:基于掩码自编码器的行人重识别预训练

PersonMAE: Person Re-Identification Pre-Training with Masked AutoEncoders

University of Science and Technology of China(中国科学技术大学);Microsoft Research Asia(微软亚洲研究院);Microsoft Cloud AI(微软云AI)

Hezhen Hu, Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Lu Yuan, Dong Chen, Houqiang Li

AI总结:

针对行人重识别特征表示的多层级感知、遮挡鲁棒、跨区域不变需求,提出融入分块掩码、双区域双层面预测的PersonMAE预训练框架,在多类ReID任务上取得SOTA性能。

摘要:

预训练在学习行人重识别(ReID)通用特征表示方面发挥着日益重要的作用。我们认为,高质量的ReID表示应具备三种属性,即多层级感知性、遮挡鲁棒性和跨区域不变性。为此,我们提出了一种简单却有效的预训练框架——PersonMAE,该框架在掩码自编码器中融入了两项核心设计,以更好地服务于行人重识别任务。1)PersonMAE从给定图像生成两个区域,以RegionA为输入,RegionB为预测目标。对RegionA施加分块掩码以模拟ReID中常见的遮挡情况,并将其剩余可见部分输入编码器。2)随后PersonMAE旨在从像素层面和语义特征层面预测整个RegionB。这一过程促使其预训练特征表示具备上述三种属性。这些属性使得PersonMAE能够适配下游行人重识别任务,在四项下游ReID任务上取得了当前最优性能,即有监督(整体和遮挡设置)以及无监督(UDA和USL设置)。值得注意的是,在常用的有监督设置下,采用ViT-B骨干网络的PersonMAE在MSMT17和OccDuke数据集上分别达到79.8%和69.5%的mAP,分别以+8.0 mAP和+5.3 mAP的大幅优势超越此前的最优水平。

英文摘要:

Pre-training is playing an increasingly important role in learning generic feature representation for Person Re-identification (ReID). We argue that a high-quality ReID representation should have three properties, namely, multi-level awareness, occlusion robustness, and cross-region invariance. To this end, we propose a simple yet effective pre-training framework, namely PersonMAE, which involves two core designs into masked autoencoders to better serve the task of Person Re-ID. 1) PersonMAE generates two regions from the given image with RegionA as the input and \textit{RegionB} as the prediction target. RegionA is corrupted with block-wise masking to mimic common occlusion in ReID and its remaining visible parts are fed into the encoder. 2) Then PersonMAE aims to predict the whole RegionB at both pixel level and semantic feature level. It encourages its pre-trained feature representations with the three properties mentioned above. These properties make PersonMAE compatible with downstream Person ReID tasks, leading to state-of-the-art performance on four downstream ReID tasks, i.e., supervised (holistic and occluded setting), and unsupervised (UDA and USL setting). Notably, on the commonly adopted supervised setting, PersonMAE with ViT-B backbone achieves 79.8% and 69.5% mAP on the MSMT17 and OccDuke datasets, surpassing the previous state-of-the-art by a large margin of +8.0 mAP, and +5.3 mAP, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑