Accurate and real-time hand gesture recognition is essential for controlling advanced hand prostheses. Surface Electromyography (sEMG) signals obtained from the forearm are widely used for this purpose. Here, we introduce a novel hand gesture representation called Temporal Muscle Activation (TMA) maps which captures information about the activation patterns of muscles in the forearm. Based on these maps, we propose an algorithm that can recognize hand gestures in real-time using a Convolution Neural Network. The algorithm was tested on 8 healthy subjects with sEMG signals acquired from 8 electrodes placed along the circumference of the forearm. The average classification accuracy of the proposed method was 94%, which is comparable to state-of-the-art methods. The average computation time of a prediction was 5.5ms, making the algorithm ideal for the real-time gesture recognition applications.
Muscle fatigue is usually defined as a decrease in the ability to produce force. The surface electromyography (sEMG) signals have been widely used to provide information about muscle activities including detecting muscle fatigue by various data-driven techniques such as machine learning and statistical approaches. However, it is well-known that sEMG signals are weak signals (low amplitude of the signals) with a low signal-to-noise ratio, data-driven techniques cannot work well when the quality of the data is poor. In particular, the existing methods are unable to detect muscle fatigue coming f…
From Continuous sEMG Signals to Discrete Muscle State Tokens: A Robust and Interpretable Representation Framework
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（北京邮电大学计算机学院（国家示范性软件工程学院））；Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education（教育部可信分布式计算与服务重点实验室（北京邮电大学））；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Institute for Precision Medicine, Tsinghua University（清华大学精准医学研究所）；College of AI, Tsinghua University（清华大学人工智能学院）；Research Academy of Grand Health, Ningbo University（宁波大学大健康研究院）；School of Information Technology and Artificial Intelligence, Zhejiang University of Finance and Economics（浙江财经大学信息技术与人工智能学院）
Recent advancements in diagnostic learning and development of gesture-based human machine interfaces have driven surface electromyography (sEMG) towards significant importance. Analysis of hand gestures requires an accurate assessment of sEMG signals. The proposed work presents a novel sequential master-slave architecture consisting of deep neural networks (DNNs) for classification of signs from the Indian sign language using signals recorded from multiple sEMG channels. The performance of the master-slave network is augmented by leveraging additional synthetic feature data generated by long s…
JetBrains Research（JetBrains研究院）；University of York（约克大学）；National Research University Higher School of Economics（国家研究型高等经济学院）
Movement control of artificial limbs has made big advances in recent years. New sensor and control technology enhanced the functionality and usefulness of artificial limbs to the point that complex movements, such as grasping, can be performed to a limited extent. To date, the most successful results were achieved by applying recurrent neural networks (RNNs). However, in the domain of artificial hands, experiments so far were limited to non-mobile wrists, which significantly reduces the functionality of such prostheses. In this paper, for the first time, we present empirical results on gesture…
Concordia University（康考迪亚大学）；New York University（纽约大学）；University of Calgary（卡尔加里大学）
Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a compact deep learning framework referred to as the CT-HGR, which employs a vision transformer network to conduct hand gesture recognition using highdensity sEMG (HD-sEMG) signals. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while deali…
Myoelectric control is one of the leading areas of research in the field of robotic prosthetics. We present our research in surface electromyography (sEMG) signal classification, where our simple and novel attention-based approach now leads the industry, universally beating more complex, state-of-the-art models. Our novel attention-based model achieves benchmark leading results on multiple industry-standard datasets including 53 finger, wrist, and grasping motions, improving over both sophisticated signal processing and CNN-based approaches. Our strong results with a straightforward model also…
Artificial Intelligence Thrust, HKUST (Guangzhou)（香港科技大学（广州）人工智能方向）；School of Computing and Information Technology, Great Bay University（湾区大学计算机与信息技术学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Department of Computer Science and Engineering, HKUST（香港科技大学计算机科学与工程系）；Guangzhou HKUST Fok Ying Tung Research Institute（广州香港科技大学霍英东研究院）
Università di Bologna（博洛尼亚大学）；Politecnico di Torino（都灵理工大学）；ETH Zurich（苏黎世联邦理工学院）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
Human-machine interaction is gaining traction in rehabilitation tasks, such as controlling prosthetic hands or robotic arms. Gesture recognition exploiting surface electromyographic (sEMG) signals is one of the most promising approaches, given that sEMG signal acquisition is non-invasive and is directly related to muscle contraction. However, the analysis of these signals still presents many challenges since similar gestures result in similar muscle contractions. Thus the resulting signal shapes are almost identical, leading to low classification accuracy. To tackle this challenge, complex neu…
Concordia University（康考迪亚大学）；Concordia Institute for Information System Engineering (CIISE)（康考迪亚大学信息系统工程研究所（CIISE））；Electrical and Computer Engineering (ECE), Concordia University（康考迪亚大学电气与计算机工程系（ECE））；Department of Bioengineering, Imperial College London（帝国理工学院生物工程系）；Electrical and Computer Engineering and Mechanical and Aerospace Engineering, New York University (NYU)（纽约大学（NYU）电气与计算机工程及机械与航空航天工程系）；New York University (NYU)（纽约大学（NYU））；Imperial College London（帝国理工学院）
There has been a surge of recent interest in Machine Learning (ML), particularly Deep Neural Network (DNN)-based models, to decode muscle activities from surface Electromyography (sEMG) signals for myoelectric control of neurorobotic systems. DNN-based models, however, require large training sets and, typically, have high structural complexity, i.e., they depend on a large number of trainable parameters. To address these issues, we developed a framework based on the Transformer architecture for processing sEMG signals. We propose a novel Vision Transformer (ViT)-based neural network architectu…
Statistical models of Surface electromyography (sEMG) signals have several applications such as better understanding of sEMG signal generation, improved pattern recognition based control of wearable exoskeletons and prostheses, improving training strategies in sports activities, and EMG simulation studies. Most of the existing studies analysed the statistical model of sEMG signals acquired under isometric contractions. However, there is no study that addresses the statistical model under isotonic contractions. In this work, a new dataset, electromyography analysis of human activities - databas…
Physics-informed Deep Learning for Muscle Force Prediction with Unlabeled sEMG Signals
School of Electronic and Electrical Engineering, University of Leeds（利兹大学电子与电气工程学院）；Centre for Wireless Innovation, Queen’s University Belfast（贝尔法斯特女王大学无线创新中心）；School of Mechanical Engineering, Tianjin University（天津大学机械工程学院）；International Institute for Innovative Design and Intelligent Manufacturing of Tianjin University in Zhejiang（天津大学浙江创新设计与智能制造国际研究院）；Rex Bionics Ltd（Rex Bionics有限公司）
针对肌肉力预测依赖标签且物理模型计算慢的问题，提出将 Hill 前向动力学作为损失嵌入 FNN 的物理信息深度学习方法，利用无标签 sEMG 实现准确的肌肉力预测与个性化肌肉-肌腱参数识别。
A Deep Learning Sequential Decoder for Transient High-Density Electromyography in Hand Gesture Recognition Using Subject-Embedded Transfer Learning
University of California Los Angeles (UCLA)（加州大学洛杉矶分校（UCLA））；New York University (NYU)（纽约大学（NYU））；NYU WIRELESS（纽约大学无线研究中心）；NYU Center for Urban Science and Progress (CUSP)（纽约大学城市科学与进步中心（CUSP））
The surface electromyography (sEMG) analysis can provide information on muscle fatigue status by estimation of muscle fibre conduction velocity (MFCV), a measure of the travelling speed of motor unit action potentials in muscle tissue. This paper proposes a technique for MFCV estimation using cross-correlation methods and variable bitstream modulation. The technique displays an estimate based on a set of data generated by the gain variation modulation, providing an average estimate of the MFCV. The observed trend of MFCV decrease correlates with the fatigue state of the observed muscle. Finall…
Concordat University（康考迪亚大学）；Concordia Institute for Information System Engineering (CITIES)（康考迪亚信息系统工程研究所（CITIES））
Recently, there has been a surge of significant interest on application of Deep Learning (DL) models to autonomously perform hand gesture recognition using surface Electromyogram (sEMG) signals. DL models are, however, mainly designed to be applied on sparse sEMG signals. Furthermore, due to their complex structure, typically, we are faced with memory constraints; require large training times and a large number of training samples, and; there is the need to resort to data augmentation and/or transfer learning. In this paper, for the first time (to the best of our knowledge), we investigate and…
User Training with Error Augmentation for Electromyogram-based Gesture Classification
Northeastern University（东北大学）；Khoury College of Computer Sciences（科里计算机科学学院）；Meta Reality Labs Research（Meta现实实验室）；Institute for Experiential AI（体验式人工智能研究院）；Department of Physical Therapy, Movement, and Rehabilitation Sciences（物理治疗、运动与康复科学系）
Recently, surface electromyography (sEMG) emerged as a novel biometric authentication method. Since EMG system parameters, such as the feature extraction methods and the number of channels, have been known to affect system performances, it is important to investigate these effects on the performance of the sEMG-based biometric system to determine optimal system parameters. In this study, three robust feature extraction methods, Time-domain (TD) feature, Frequency Division Technique (FDT), and Autoregressive (AR) feature, and their combinations were investigated while the number of channels var…
Surface electromyography (sEMG) is becoming exceeding useful in applications involving analysis of human motion such as in human-machine interface, assistive technology, healthcare and prosthetic development. The proposed work presents a novel dual stage classification approach for classification of grasping gestures from sEMG signals. A statistical assessment of these activities is presented to determine the similar characteristics between the considered activities. Similar activities are grouped together. In the first stage of classification, an activity is identified as belonging to a group…
Convolutional neural network (CNN) has been widely exploited for simultaneous and proportional myoelectric control due to its capability of deriving informative, representative and transferable features from surface electromyography (sEMG). However, muscle contractions have strong temporal dependencies but conventional CNN can only exploit spatial correlations. Considering that long short-term memory neural network (LSTM) is able to capture long-term and non-linear dynamics of time-series data, in this paper we propose a CNNLSTM hybrid framework to fully explore the temporal-spatial informatio…
Surface electromyography (sEMG) provides an intuitive and non-invasive interface from which to control machines. However, preserving the myoelectric control system's performance over multiple days is challenging, due to the transient nature of the signals obtained with this recording technique. In practice, if the system is to remain usable, a time-consuming and periodic recalibration is necessary. In the case where the sEMG interface is employed every few days, the user might need to do this recalibration before every use. Thus, severely limiting the practicality of such a control method. Con…
Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals has recently shown significant potential for development of advanced myoelectric-controlled prosthesis. Existing deep learning approaches, typically, include only one model as such can hardly maintain acceptable generalization performance in changing scenarios. In this paper, we aim to address this challenge by capitalizing on the recent advances of hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revoluti…
National Taiwan University（国立台湾大学）；Academia Sinica（中央研究院）；National Taiwan University of Science and Technology（国立台湾科技大学）
Electrocardiogram (ECG) artifact contamination often occurs in surface electromyography (sEMG) applications when the measured muscles are in proximity to the heart. Previous studies have developed and proposed various methods, such as high-pass filtering, template subtraction and so forth. However, these methods remain limited by the requirement of reference signals and distortion of original sEMG. This study proposed a novel denoising method to eliminate ECG artifacts from the single-channel sEMG signals using fully convolutional networks (FCN). The proposed method adopts a denoise autoencode…
Integrated Sensing and Communications for Real-time Avatar Control in XR over 5G
IDLab, University of Antwerp - imec（ID实验室，安特卫普大学 - imec）；IDLab, Ghent University - imec（ID实验室，根特大学 - imec）；Lighting Technology Lab, KU Leuven（照明技术实验室，鲁汶大学）
Surface electromyography (sEMG) has gained significant importance during recent advancements in consumer electronics for healthcare systems, gesture analysis and recognition and sign language communication. For such a system, it is imperative to determine the regions of activity in a continuously recorded sEMG signal. The proposed work provides a novel activity detection approach based on Hidden Markov Models (HMM) using sEMG signals recorded when various hand gestures are performed. Detection procedure is designed based on a probabilistic outlook by making use of mathematical models. The requ…
The probability density function (pdf) of surface Electromyography (sEMG) signals follows any one of the standalone standard distributions: the Gaussian or the Laplacian. Further, the choice of the model is dependent on muscle contraction force (MCF) levels. Hence, a unified model is proposed which explains the statistical nature of sEMG signals at different MCF levels. In this paper, we propose the Laplacian Gaussian Mixture (LGM) model for the signals recorded from upper limbs. This model is able to explain the sEMG signals from different activities corresponding to different MCF levels. The…
Concordia University（康考迪亚大学）；Concordia Institute for Information System Engineering（康考迪亚信息系统工程研究院）；New York University（纽约大学）
Capitalizing on the need for addressing the existing challenges associated with gesture recognition via sparse multichannel surface Electromyography (sEMG) signals, the paper proposes a novel deep learning model, referred to as the XceptionTime architecture. The proposed innovative XceptionTime is designed by integration of depthwise separable convolutions, adaptive average pooling, and a novel non-linear normalization technique. At the heart of the proposed architecture is several XceptionTime modules concatenated in series fashion designed to capture both temporal and spatial information-bea…
SDEMG: Score-based Diffusion Model for Surface Electromyographic Signal Denoising
National Yang Ming Chiao Tung University（国立阳明交通大学）；National Taiwan University（国立台湾大学）；National Taipei University of Technology（国立台北科技大学）；National Taiwan University of Science and Technology（国立台湾科技大学）；Academia Sinica（中央研究院）
Deep convolutional neural networks (CNNs) are appealing for the purpose of classification of hand movements from surface electromyography (sEMG) data because they have the ability to perform automated person-specific feature extraction from raw data. In this paper, we make the novel contribution of proposing and evaluating a design for the early processing layers in the deep CNN for multichannel sEMG. Specifically, we propose a novel temporal-to-spatial (TtS) CNN architecture, where the first layer performs convolution separately on each sEMG channel to extract temporal features. This is motiv…
