Federal Institute of São Carlos - UFSCar（圣卡洛斯联邦学院）；Eldorado Research Institute（埃尔多拉多研究院）；São Paulo State University - UNESP（圣保罗州立大学）
Claudio Filipi Gonçalves dos Santos, Diego de Souza Oliveira, Leandro A. Passos, Rafael Gonçalves Pires, Daniel Felipe Silva Santos, Lucas Pascotti Valem, Thier… 展开作者
Claudio Filipi Gonçalves dos Santos, Diego de Souza Oliveira, Leandro A. Passos, Rafael Gonçalves Pires, Daniel Felipe Silva Santos, Lucas Pascotti Valem, Thierry P. Moreira, Marcos Cleison S. Santana, Mateus Roder, João Paulo Papa, Danilo Colombo
In general, biometry-based control systems may not rely on individual expected behavior or cooperation to operate appropriately. Instead, such systems should be aware of malicious procedures for unauthorized access attempts. Some works available in the literature suggest addressing the problem through gait recognition approaches. Such methods aim at identifying human beings through intrinsic perceptible features, despite dressed clothes or accessories. Although the issue denotes a relatively long-time challenge, most of the techniques developed to handle the problem present several drawbacks related to feature extraction and low classification rates, among other issues. However, deep learning-based approaches recently emerged as a robust set of tools to deal with virtually any image and computer-vision related problem, providing paramount results for gait recognition as well. Therefore, this work provides a surveyed compilation of recent works regarding biometric detection through gait recognition with a focus on deep learning approaches, emphasizing their benefits, and exposing their weaknesses. Besides, it also presents categorized and characterized descriptions of the datasets, approaches, and architectures employed to tackle associated constraints.
1 Shanghai Key Lab of Intelligent Information Processing, School of Computer Science 2 Institute of Science
Gait recognition plays a vital role in human identification since gait is a unique biometric feature that can be perceived at a distance. Although existing gait recognition methods can learn gait features from gait sequences in different ways, the performance of gait recognition suffers from insufficient labeled data, especially in some practical scenarios associated with short gait sequences or various clothing styles. It is unpractical to label the numerous gait data. In this work, we propose a self-supervised gait recognition method, termed SelfGait, which takes advantage of the massive, di…
A Comprehensive Survey on Deep Gait Recognition: Algorithms, Datasets and Challenges
The University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Hong Kong Polytechnic University（香港理工大学）
Gait recognition under multiple views is an important computer vision and pattern recognition task. In the emerging convolutional neural network based approaches, the information of view angle is ignored to some extent. Instead of direct view estimation and training view-specific recognition models, we propose a compatible framework that can embed view information into existing architectures of gait recognition. The embedding is simply achieved by a selective projection layer. Experimental results on two large public datasets show that the proposed framework is very effective.
Hangzhou Dianzi University（杭州电子科技大学）；Explore Academy of JD.com（京东探索研究院）
Existing studies for gait recognition are dominated by 2D representations like the silhouette or skeleton of the human body in constrained scenes. However, humans live and walk in the unconstrained 3D space, so projecting the 3D human body onto the 2D plane will discard a lot of crucial information like the viewpoint, shape, and dynamics for gait recognition. Therefore, this paper aims to explore dense 3D representations for gait recognition in the wild, which is a practical yet neglected problem. In particular, we propose a novel framework to explore the 3D Skinned Multi-Person Linear (SMPL) …
Southern University of Science and Technology（南方科技大学）；Research Institute of Trustworthy Autonomous System, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；The University of Hong Kong（香港大学）；Beijing Normal University（北京师范大学）
Gait recognition is one of the most critical long-distance identification technologies and increasingly gains popularity in both research and industry communities. Despite the significant progress made in indoor datasets, much evidence shows that gait recognition techniques perform poorly in the wild. More importantly, we also find that some conclusions drawn from indoor datasets cannot be generalized to real applications. Therefore, the primary goal of this paper is to present a comprehensive benchmark study for better practicality rather than only a particular model for better performance. T…
Human gait is one of important biometric characteristics for human identification at a distance. In practice, occlusion usually occurs and seriously affects accuracy of gait recognition. However, there is no available database to support in-depth research of this problem, and state-of-arts gait recognition methods have not paid enough attention to it, thus this paper focuses on gait recognition under occlusion. We collect a new gait recognition database called OG RGB+D database, which breaks through the limitation of other gait databases and includes multimodal gait data of various occlusions …
Hebei Key Laboratory for Electromagnetic Environmental Effects and Information Processing（河北省电磁环境效应与信息处理重点实验室）；Shijiazhuang Tiedao University（石家庄铁道大学）
Gait recognition, which can realize long-distance and contactless identification, is an important biometric technology. Recent gait recognition methods focus on learning the pattern of human movement or appearance during walking, and construct the corresponding spatio-temporal representations. However, different individuals have their own laws of movement patterns, simple spatial-temporal features are difficult to describe changes in motion of human parts, especially when confounding variables such as clothing and carrying are included, thus distinguishability of features is reduced. In this p…
National Research University Higher School of Economics（国立研究型大学高等经济学院）；Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）
Human gait or walking manner is a biometric feature that allows identification of a person when other biometric features such as the face or iris are not visible. In this paper, we present a new pose-based convolutional neural network model for gait recognition. Unlike many methods that consider the full-height silhouette of a moving person, we consider the motion of points in the areas around human joints. To extract motion information, we estimate the optical flow between consecutive frames. We propose a deep convolutional model that computes pose-based gait descriptors. We compare different…
GaitKD: A Universal Decoupled Distillation Framework for Efficient Gait Recognition
The City University of New York（纽约城市大学）；Wuhan University（武汉大学）；Beijing Jiaotong University（北京交通大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Oulu（奥卢大学）
Whole-body-based human authentication is a promising approach for remote biometrics scenarios. Current literature focuses on either body recognition based on RGB images or gait recognition based on body shapes and walking patterns; both have their advantages and drawbacks. In this work, we propose Dual-Modal Ensemble (DME), which combines both RGB and silhouette data to achieve more robust performances for indoor and outdoor whole-body based recognition. Within DME, we propose GaitPattern, which is inspired by the double helical gait pattern used in traditional gait analysis. The GaitPattern c…
GaitASMS: Gait Recognition by Adaptive Structured Spatial Representation and Multi-Scale Temporal Aggregation
School of Computer Engineering and Science, Shanghai University（上海大学计算机工程与科学学院）；School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）
College of Computer Science and Technology, Qingdao University（青岛大学计算机科学技术学院）；College of Information Science and Engineering, Ocean University of China（中国海洋大学信息科学与工程学院）；School of Informatics, University of Leicester（莱斯特大学信息学院）
It is a challenging task to identify a person based on her/his gait patterns. State-of-the-art approaches rely on the analysis of temporal or spatial characteristics of gait, and gait recognition is usually performed on single modality data (such as images, skeleton joint coordinates, or force signals). Evidence has shown that using multi-modality data is more conducive to gait research. Therefore, we here establish an automated learning system, with an associated spatio-temporal capsule network (ASTCapsNet) trained on multi-sensor datasets, to analyze multimodal information for gait recogniti…
DAGait: Generalized Skeleton-Guided Data Alignment for Gait Recognition
The Shenzhen International Graduate School, Tsinghua University Shenzhen, China（深圳国际研究生院，清华大学深圳学院中国）；School of Computer Science（计算机科学学院）；Information Engineering, Hefei University of Technology Hefei, China（信息工程，合肥工业大学合肥中国）
Hangzhou Dianzi University（杭州电子科技大学）；JD Explore Academy（京东探索研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Lishui Institute of Hangzhou Dianzi University（杭州电子科技大学丽水研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Existing studies for gait recognition are dominated by in-the-lab scenarios. Since people live in real-world senses, gait recognition in the wild is a more practical problem that has recently attracted the attention of the community of multimedia and computer vision. Current methods that obtain state-of-the-art performance on in-the-lab benchmarks achieve much worse accuracy on the recently proposed in-the-wild datasets because these methods can hardly model the varied temporal dynamics of gait sequences in unconstrained scenes. Therefore, this paper presents a novel multi-hop temporal switch …
Gait recognition, which identifies individuals based on their walking patterns, is an important biometric technique since it can be observed from a distance and does not require the subject's cooperation. Recognizing a person's gait is difficult because of the appearance variants in human silhouette sequences produced by varying viewing angles, carrying objects, and clothing. Recent research has produced a number of ways for coping with these variants. In this paper, we present the usage of inferring 3-D body shapes distilled from limited images, which are, in principle, invariant to the speci…
There are several confounding factors that can reduce the accuracy of gait recognition systems. These factors can reduce the distinctiveness, or alter the features used to characterise gait, they include variations in clothing, lighting, pose and environment, such as the walking surface. Full invariance to all confounding factors is challenging in the absence of high-quality labelled training data. We introduce a simulation-based methodology and a subject-specific dataset which can be used for generating synthetic video frames and sequences for data augmentation. With this methodology, we gene…
Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；The Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）；School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）；Southern University of Science and Technology（南方科技大学）；Watrix Technology Limited Co. Ltd.（华瑞芯科技有限公司）
Zhejiang University（浙江大学）；Merit Interactive Co Ltd.（默瑞互动有限公司）；Shanghai Institute for Advanced Study Zhejiang University（浙江大学上海高等研究院）
With the motivation of practical gait recognition applications, we propose to automatically create a large-scale synthetic gait dataset (called VersatileGait) by a game engine, which consists of around one million silhouette sequences of 11,000 subjects with fine-grained attributes in various complicated scenarios. Compared with existing real gait datasets with limited samples and simple scenarios, the proposed VersatileGait dataset possesses several nice properties, including huge dataset size, high sample diversity, high-quality annotations, multi-pitch angles, small domain gap with the real…
OpenGait: A Comprehensive Benchmark Study for Gait Recognition towards Better Practicality
School of Artificial Intelligence, Shenzhen University (SZU)（深圳大学人工智能学院）；National Engineering Laboratory for Big Data System Computing Technology, SZU（大数据系统计算技术国家工程实验室）；Southern University of Science and Technology (SUSTech)（南方科技大学）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Watrix Technology Limited Co. Ltd（纬思科技有限公司）；Department of Computer Vision Technology (VIS), Baidu Inc.（百度公司计算机视觉技术部）；Shenzhen Institute of Advanced Study, University of Electronic Science and Technology of China (UESTC)（中国电子科学研究院深圳研究院）；Institute of Applied Artificial Intelligence of the Guangdong-Hong Kong-Macao Greater Bay Area, Shenzhen Polytechnic University（粤港澳大湾区应用人工智能研究院，深圳职业技术大学）；Department of Computer Science and Engineering, Southern University of Science and Technology (SUSTech)（南方科技大学计算机科学与工程系）
Unsupervised Domain Adaptation with Dynamic Clustering and Contrastive Refinement for Gait Recognition
School of Computer Engineering and Science, Shanghai University（上海大学计算机工程与科学学院）；School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）
Improving the generalization of gait recognition with limited datasets
School of Computer Science, Wuhan University（武汉大学计算机学院）；Department of Automation, School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学自动化系）；Shenzhen Institute of Advanced Study, University of Electronic Science and Technology of China (UESTC)（电子科技大学深圳研究院）；Department of Computer Science and Engineering, Southern University of Science and Technology (SUSTech)（南方科技大学计算机科学与工程系）
The use of gait for person identification has important advantages such as being non-invasive, unobtrusive, not requiring cooperation and being less likely to be obscured compared to other biometrics. Existing methods for gait recognition require cooperative gait scenarios, in which a single person is walking multiple times in a straight line in front of a camera. We aim to address the challenges of real-world scenarios in which camera feeds capture multiple people, who in most cases pass in front of the camera only once. We address privacy concerns by using only motion information of walking …
Learning Gait Representation from Massive Unlabelled Walking Videos: A Benchmark
Southern University of Science and Technology（南方科技大学）；Beijing Normal University（北京师范大学）；Watrix Technology Limited Co. Ltd.（银河水滴科技有限公司）；University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Idaho National Laboratory（爱达荷国家实验室）；University at Buffalo（布法罗大学）
Gait is a person's natural walking style and a complex biological process that is unique to each person. Recently, the channel state information (CSI) of WiFi devices have been exploited to capture human gait biometrics for user identification. However, the performance of existing CSI-based gait identification systems is far from satisfactory. They can only achieve limited identification accuracy (maximum $93\%$) only for a very small group of people (i.e., between 2 to 10). To address such challenge, an end-to-end deep CSI learning system is developed, which exploits deep neural networks to a…
While the Vision Transformer has been used in gait recognition, its application in multi-view gait recognition is still limited. Different views significantly affect the extraction and identification accuracy of the characteristics of gait contour. To address this, this paper proposes a Siamese Mobile Vision Transformer (SMViT). This model not only focuses on the local characteristics of the human gait space but also considers the characteristics of long-distance attention associations, which can extract multi-dimensional step status characteristics. In addition, it describes how different per…
Zhejiang University（浙江大学）；Key Laboratory of Collaborative Sensing and Autonomous Unmanned Systems of Zhejiang Province（浙江省协同感知与自主无人系统重点实验室）
Gait recognition is widely used in diversified practical applications. Currently, the most prevalent approach is to recognize human gait from RGB images, owing to the progress of computer vision technologies. Nevertheless, the perception capability of RGB cameras deteriorates in rough circumstances, and visual surveillance may cause privacy invasion. Due to the robustness and non-invasive feature of millimeter wave (mmWave) radar, radar-based gait recognition has attracted increasing attention in recent years. In this research, we propose a Hierarchical Dynamic Network (HDNet) for gait recogni…
University of Central Florida（中佛罗里达大学）；Creighton University（克瑞顿大学）
In this paper, we propose a novel gait recognition method based on a bag-of-words feature representation method. The algorithm is trained, tested and evaluated on a unique human gait data consisting of 93 individuals who walked with comfortable pace between two end points during two different sessions. To evaluate the effectiveness of the proposed model, the results are compared with the outputs of the classification using extracted features. As it is presented, the proposed method results in significant improvement accuracy compared to using common statistical features, in all the used classi…
Indian Institute of Technology (BHU)（印度理工学院（贝纳拉斯印度教大学））；Nanyang Technological University（南洋理工大学）
Recent advancements in computational resources and Deep Learning methodologies has significantly benefited development of intelligent vision-based surveillance applications. Gait recognition in the presence of occlusion is one of the challenging research topics in this area, and the solutions proposed by researchers to date lack in robustness and also dependent of several unrealistic constraints, which limits their practical applicability. We improve the state-of-the-art by developing novel deep learning-based algorithms to identify the occluded frames in an input sequence and next reconstruct…
Babes–Bolyai University（巴贝什-博雅依大学）；Sapientia Hungarian University of Transylvania（特兰西瓦尼亚匈牙利语大学）
Recent advances in pattern matching, such as speech or object recognition support the viability of feature learning with deep learning solutions for gait recognition. Past papers have evaluated deep neural networks trained in a supervised manner for this task. In this work, we investigated both supervised and unsupervised approaches. Feature extractors using similar architectures incorporated into end-to-end models and autoencoders were compared based on their ability of learning good representations for a gait verification system. Both feature extractors were trained on the IDNet dataset then…
Gait recognition, referring to the identification of individuals based on the manner in which they walk, can be very challenging due to the variations in the viewpoint of the camera and the appearance of individuals. Current methods for gait recognition have been dominated by deep learning models, notably those based on partial feature representations. In this context, we propose a novel deep network, learning to transfer multi-scale partial gait representations using capsules to obtain more discriminative gait features. Our network first obtains multi-scale partial representations using a sta…
Gait recognition is a promising video-based biometric for identifying individual walking patterns from a long distance. At present, most gait recognition methods use silhouette images to represent a person in each frame. However, silhouette images can lose fine-grained spatial information, and most papers do not regard how to obtain these silhouettes in complex scenes. Furthermore, silhouette images contain not only gait features but also other visual clues that can be recognized. Hence these approaches can not be considered as strict gait recognition. We leverage recent advances in human pose…
In this paper, a performance evaluation of well-known deep learning models in gait recognition is presented. For this purpose, the transfer learning scheme is adopted to pre-trained models in order to fit the models to the CASIA-B dataset for solving a gait recognition task. In this context, 18 popular Convolutional Neural Networks (CNNs), were re-trained using Gait Energy Images (GEIs) of CASIA-B containing almost 14000 images of 124 classes under various conditions, and their performance was studied in terms of accuracy. Moreover, the performance of the studied models is managed to be explai…
Gait recognition refers to the identification of individuals based on features acquired from their body movement during walking. Despite the recent advances in gait recognition with deep learning, variations in data acquisition and appearance, namely camera angles, subject pose, occlusions, and clothing, are challenging factors that need to be considered for achieving accurate gait recognition systems. In this paper, we propose a network that first learns to extract gait convolutional energy maps (GCEM) from frame-level convolutional features. It then adopts a bidirectional recurrent neural ne…
Skeleton-based gait recognition models usually suffer from the robustness problem, as the Rank-1 accuracy varies from 90\% in normal walking cases to 70\% in walking with coats cases. In this work, we propose a state-of-the-art robust skeleton-based gait recognition model called Gait-TR, which is based on the combination of spatial transformer frameworks and temporal convolutional networks. Gait-TR achieves substantial improvements over other skeleton-based gait models with higher accuracy and better robustness on the well-known gait dataset CASIA-B. Particularly in walking with coats cases, G…
