基于补丁混合跨模态学习的可见光-红外行人重识别
Visible-Infrared Person Re-Identification via Patch-Mixed Cross-Modality Learning
AI总结：
本文提出补丁混合跨模态框架PMCM，通过将同一行人可见光与红外图像切块拼接并引入部件对齐与模态学习损失，缓解模态差异与不平衡问题，在VI-ReID任务上取得最优性能。
摘要：
可见光-红外行人重识别（VI-ReID）旨在从不同模态中检索同一行人的图像，其挑战在于显著的模态差异。为缓解模态差距，近期方法通过生成对抗网络（GANs）、灰度化或混合（mixup）策略生成中间图像。然而，这些方法可能引入额外的数据分布，且两种模态之间的语义对应关系未能得到充分学习。本文提出了一种补丁混合跨模态框架（PMCM），将来自两种模态的同一行人的两张图像分割成补丁并拼接成一张新图像用于模型学习。引入部件对齐损失以正则化表示学习，并提出补丁混合模态学习损失以对齐模态间关系。通过这种方式，模型学习通过不同风格的补丁识别行人，从而能够推断模态间的语义对应关系。此外，借助灵活的图像生成策略，补丁混合图像可自由调整不同模态补丁的比例，从而进一步缓解模态不平衡问题。在两个VI-ReID数据集上，所提方法取得了新的最先进性能。
英文摘要：
Visible-infrared person re-identification (VI-ReID) aims to retrieve images of the same pedestrian from different modalities, where the challenges lie in the significant modality discrepancy. To alleviate the modality gap, recent methods generate intermediate images by GANs, grayscaling, or mixup strategies. However, these methods could introduce extra data distribution, and the semantic correspondence between the two modalities is not well learned. In this paper, we propose a Patch-Mixed Cross-Modality framework (PMCM), where two images of the same person from two modalities are split into patches and stitched into a new one for model learning. A part-alignment loss is introduced to regularize representation learning, and a patch-mixed modality learning loss is proposed to align between the modalities. In this way, the model learns to recognize a person through patches of different styles, thereby the modality semantic correspondence can be inferred. In addition, with the flexible image generation strategy, the patch-mixed images freely adjust the ratio of different modality patches, which could further alleviate the modality imbalance problem. On two VI-ReID datasets, we report new state-of-the-art performance with the proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
