arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.08387cs.CV
收藏

面向换衣行人重识别的语义感知注意力与视觉遮挡网络

A Semantic-aware Attention and Visual Shielding Network for Cloth-changing Person Re-identification

Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin University of Technology(天津理工大学计算机视觉与系统教育部重点实验室);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院)

Zan Gao, Hongwei Wei, Weili Guan, Jie Nie, Meng Wang, Shenyong Chen

AI总结:

提出语义感知注意力与视觉遮挡网络(SAVS),通过屏蔽衣服区域并聚焦人体语义信息,解决换衣行人重识别特征鲁棒性问题,在LTCC和PRCC数据集上显著优于现有方法。

摘要:

换衣行人重识别(ReID)是一个新兴的研究课题,旨在检索换衣后的行人。由于不同衣服下的人体外貌存在较大差异,现有方法很难提取具有判别力和鲁棒性的特征表示。当前工作主要关注体型或轮廓草图,但人体语义信息以及换衣前后行人特征的潜在一致性尚未被充分探索或被忽略。为解决这些问题,本文提出了一种新颖的面向换衣行人重识别的语义感知注意力与视觉遮挡网络(简称SAVS),其核心思想是屏蔽与衣服外观相关的线索,仅关注对视点/姿态变化不敏感的视觉语义信息。具体而言,首先采用视觉语义编码器,基于人体语义分割信息定位人体和衣服区域。然后,提出人体语义注意力模块(HSA),以突出人体语义信息并重新加权视觉特征图。此外,还设计了视觉衣服遮挡模块(VCS),通过覆盖衣服区域,使模型聚焦于与衣服无关的视觉语义信息,从而为换衣任务提取更鲁棒的特征表示。最重要的是,这两个模块在端到端统一框架中被联合探索。大量实验表明,所提方法能显著优于现有最先进方法,并能提取更鲁棒的换衣行人特征。与FSAM(发表于CVPR 2021)相比,该方法在LTCC和PRCC数据集上的mAP(rank-1)分别提升了32.7%(16.5%)和14.9%(-)。

英文摘要:

Cloth-changing person reidentification (ReID) is a newly emerging research topic that aims to retrieve pedestrians whose clothes are changed. Since the human appearance with different clothes exhibits large variations, it is very difficult for existing approaches to extract discriminative and robust feature representations. Current works mainly focus on body shape or contour sketches, but the human semantic information and the potential consistency of pedestrian features before and after changing clothes are not fully explored or are ignored. To solve these issues, in this work, a novel semantic-aware attention and visual shielding network for cloth-changing person ReID (abbreviated as SAVS) is proposed where the key idea is to shield clues related to the appearance of clothes and only focus on visual semantic information that is not sensitive to view/posture changes. Specifically, a visual semantic encoder is first employed to locate the human body and clothing regions based on human semantic segmentation information. Then, a human semantic attention module (HSA) is proposed to highlight the human semantic information and reweight the visual feature map. In addition, a visual clothes shielding module (VCS) is also designed to extract a more robust feature representation for the cloth-changing task by covering the clothing regions and focusing the model on the visual semantic information unrelated to the clothes. Most importantly, these two modules are jointly explored in an end-to-end unified framework. Extensive experiments demonstrate that the proposed method can significantly outperform state-of-the-art methods, and more robust features can be extracted for cloth-changing persons. Compared with FSAM (published in CVPR 2021), this method can achieve improvements of 32.7% (16.5%) and 14.9% (-) on the LTCC and PRCC datasets in terms of mAP (rank-1), respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑