探索多种步态模态以提升人体识别性能
Exploring More from Multiple Gait Modalities for Human Identification
AI总结：
本研究系统比较轮廓、人体解析和光流三种步态模态的表示能力与融合策略，提出C$^2$Fusion方法构建MultiGait++框架，在多个数据集上验证其有效性。
摘要：
步态作为一种软生物特征，能够在远距离反映个体独特的行走模式，展现出用于无约束人体识别的有前景的技术。通过大幅排除RGB视频中隐藏的与步态无关的线索，轮廓和骨骼虽然视觉上紧凑，但长期以来一直作为两种最流行的步态模态。近期，一些尝试引入了更具信息量的数据形式，如人体解析和光流图像，以捕捉步态特征，并采用了多分支架构。然而，由于模型设计和实验设置中的不一致性，我们认为在这些流行步态模态之间，涉及表示能力和融合策略探索的全面且公平的比较研究仍然缺乏。从细粒度与粗粒度形状以及整体与像素级运动建模的角度，本工作对三种流行的步态表示（即轮廓、人体解析和光流）进行了深入研究，并进行了多种融合评估，实验揭示了它们的相似性和差异性。基于获得的见解，我们进一步开发了一种C$^2$Fusion策略，从而构建了我们的新框架MultiGait++。C$^2$Fusion在保留共性的同时突出差异，以丰富步态特征的学习。为了验证我们的发现和结论，我们在Gait3D、GREW、CCPG和SUSTech1K上进行了大量实验。代码可在https://github.com/ShiqiYu/OpenGait获取。
英文摘要：
The gait, as a kind of soft biometric characteristic, can reflect the distinct walking patterns of individuals at a distance, exhibiting a promising technique for unrestrained human identification. With largely excluding gait-unrelated cues hidden in RGB videos, the silhouette and skeleton, though visually compact, have acted as two of the most prevailing gait modalities for a long time. Recently, several attempts have been made to introduce more informative data forms like human parsing and optical flow images to capture gait characteristics, along with multi-branch architectures. However, due to the inconsistency within model designs and experiment settings, we argue that a comprehensive and fair comparative study among these popular gait modalities, involving the representational capacity and fusion strategy exploration, is still lacking. From the perspectives of fine vs. coarse-grained shape and whole vs. pixel-wise motion modeling, this work presents an in-depth investigation of three popular gait representations, i.e., silhouette, human parsing, and optical flow, with various fusion evaluations, and experimentally exposes their similarities and differences. Based on the obtained insights, we further develop a C$^2$Fusion strategy, consequently building our new framework MultiGait++. C$^2$Fusion preserves commonalities while highlighting differences to enrich the learning of gait features. To verify our findings and conclusions, extensive experiments on Gait3D, GREW, CCPG, and SUSTech1K are conducted. The code is available at https://github.com/ShiqiYu/OpenGait.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
