Combo-Gait:多模态和多任务框架用于多模态步态识别与属性分析
Combo-Gait: Unified Transformer Framework for Multi-Modal Gait Recognition and Attribute Analysis
AI总结:
本文提出结合2D时间轮廓与3D SMPL特征的多模态框架,实现步态识别与年龄、BMI、性别等属性估计,通过统一Transformer融合多模态特征,提升鲁棒性和准确性。
摘要:
步态识别是远程人类识别的重要生物特征,尤其在低分辨率或非约束环境下。当前研究通常聚焦于2D或3D表示,但单一模态难以捕捉人体行走模式的完整几何和动态复杂性。本文提出一个多模态和多任务框架,结合2D时间轮廓与3D SMPL特征进行稳健的步态分析。除了识别外,引入多任务学习策略,联合执行步态识别和人类属性估计,包括年龄、BMI和性别。采用统一Transformer有效融合多模态步态特征,更好地学习属性相关表示,同时保留判别性身份线索。在大规模BRIAR数据集上的大量实验表明,本文方法在步态识别上优于现有最先进方法,并提供准确的人类属性估计。这些结果突显了多模态和多任务学习在推进真实场景中基于步态的人类理解方面的潜力。
英文摘要:
Gait recognition is an important biometric for human identification at a distance, particularly under low-resolution or unconstrained environments. Current works typically focus on either 2D representations (e.g., silhouettes and skeletons) or 3D representations (e.g., meshes and SMPLs), but relying on a single modality often fails to capture the full geometric and dynamic complexity of human walking patterns. In this paper, we propose a multi-modal and multi-task framework that combines 2D temporal silhouettes with 3D SMPL features for robust gait analysis. Beyond identification, we introduce a multitask learning strategy that jointly performs gait recognition and human attribute estimation, including age, body mass index (BMI), and gender. A unified transformer is employed to effectively fuse multi-modal gait features and better learn attribute-related representations, while preserving discriminative identity cues. Extensive experiments on the large-scale BRIAR datasets, collected under challenging conditions such as long-range distances (up to 1 km) and extreme pitch angles (up to 50°), demonstrate that our approach outperforms state-of-the-art methods in gait recognition and provides accurate human attribute estimation. These results highlight the promise of multi-modal and multitask learning for advancing gait-based human understanding in real-world scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。