Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a compact deep learning framework referred to as the CT-HGR, which employs a vision transformer network to conduct hand gesture recognition using highdensity sEMG (HD-sEMG) signals. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. CT-HGR can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the CT-HGR framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the CT-HGR is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed CT-HGR framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The CT-HGR achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image.
Concordat University（康考迪亚大学）；Concordia Institute for Information System Engineering (CITIES)（康考迪亚信息系统工程研究所（CITIES））
Recently, there has been a surge of significant interest on application of Deep Learning (DL) models to autonomously perform hand gesture recognition using surface Electromyogram (sEMG) signals. DL models are, however, mainly designed to be applied on sparse sEMG signals. Furthermore, due to their complex structure, typically, we are faced with memory constraints; require large training times and a large number of training samples, and; there is the need to resort to data augmentation and/or transfer learning. In this paper, for the first time (to the best of our knowledge), we investigate and…
School of Physical Science and Technology, Lanzhou University（兰州大学物理科学与技术学院）；School of Advanced Technology, Xi'an Jiaotong-Liverpool University（西交利物浦大学先进技术学院）；Chongqing Jiangbei Bachuan Quantum Middle School（重庆江北巴川量子中学）；School of Chengdu Shishi Cambridge A-Level Centre（成都石室剑桥高级课程中心）
This work introduces a method for high-accuracy EMG based gesture identification. A newly developed deep learning method, namely, deep residual shrinkage network is applied to perform gesture identification. Based on the feature of EMG signal resulting from gestures, optimizations are made to improve the identification accuracy. Finally, three different algorithms are applied to compare the accuracy of EMG signal recognition with that of DRSN. The result shows that DRSN excel traditional neural networks in terms of EMG recognition accuracy. This paper provides a reliable way to classify EMG si…
Berkeley Wireless Research Center, EECS Department, University of California, Berkeley（加州大学伯克利分校电气工程与计算机科学系伯克利无线研究中心）；Integrated System Laboratory, ETH Zurich（苏黎世联邦理工学院集成系统实验室）；DEI, University of Bologna（博洛尼亚大学DEI（可能是Department of Electrical Engineering and Information Technology的缩写，常见译为电气工程与信息技术系））
EMG-based gesture recognition shows promise for human-machine interaction. Systems are often afflicted by signal and electrode variability which degrades performance over time. We present an end-to-end system combating this variability using a large-area, high-density sensor array and a robust classification algorithm. EMG electrodes are fabricated on a flexible substrate and interfaced to a custom wireless device for 64-channel signal acquisition and streaming. We use brain-inspired high-dimensional (HD) computing for processing EMG features in one-shot learning. The HD algorithm is tolerant …
Institute of Neuroinformatics, University of Zurich and ETH Zurich（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）
The discrimination of human gestures using wearable solutions is extremely important as a supporting technique for assisted living, healthcare of the elderly and neurorehabilitation. This paper presents a mobile electromyography (EMG) analysis framework to be an auxiliary component in physiotherapy sessions or as a feedback for neuroprosthesis calibration. We implemented a framework that allows the integration of multisensors, EMG and visual information, to perform sensor fusion and to improve the accuracy of hand gesture recognition tasks. In particular, we used an event-based camera adapted …
University of California, Berkeley（加州大学伯克利分校）；University of Bologna（博洛尼亚大学）；ETH Zurich（苏黎世联邦理工学院）
Varying contraction levels of muscles is a big challenge in electromyography-based gesture recognition. Some use cases require the classifier to be robust against varying force changes, while others demand to distinguish between different effort levels of performing the same gesture. We use brain-inspired hyperdimensional computing paradigm to build classification models that are both robust to these variations and able to recognize multiple contraction levels. Experimental results on 5 subjects performing 9 gestures with 3 effort levels show up to 39.17% accuracy drop when training and testin…
University of California, Berkeley（加州大学伯克利分校）；Chan-Zuckerberg Biohub（陈·扎克伯格生物中心）
Electromyogram (EMG) pattern recognition can be used to classify hand gestures and movements for human-machine interface and prosthetics applications, but it often faces reliability issues resulting from limb position change. One method to address this is dual-stage classification, in which the limb position is first determined using additional sensors to select between multiple position-specific gesture classifiers. While improving performance, this also increases model complexity and memory footprint, making a dual-stage classifier difficult to implement in a wearable device with limited res…
Surface electromyogram (sEMG) signals result from muscle movement and hence they are an ideal candidate for benchmarking event-driven sensing and computing. We propose a simple yet novel approach for optimizing the spike encoding algorithm's hyper-parameters inspired by the readout layer concept in reservoir computing. Using a simple machine learning algorithm after spike encoding, we report performance higher than the state-of-the-art spiking neural networks on two open-source datasets for hand gesture recognition. The spike encoded data is processed through a spiking reservoir with a biologi…
Topology of surface electromyogram signals: hand gesture decoding on Riemannian manifolds
University of California, Davis（加利福尼亚大学戴维斯分校）；Center for Brain and Mind, University of California, Davis（加利福尼亚大学戴维斯分校脑与心智中心）；Department of Neurobiology, Physiology, and Behavior, University of California, Davis（加利福尼亚大学戴维斯分校神经生物学、生理学与行为学系）；Department of Otolaryngology/Head and Neck Surgery, University of California, Davis（加利福尼亚大学戴维斯分校耳鼻喉科/头颈外科）
The research in myoelectric control systems primarily focuses on extracting discriminative representations from the electromyographic (EMG) signal by designing handcrafted features. Recently, deep learning techniques have been applied to the challenging task of EMG-based gesture recognition. The adoption of these techniques slowly shifts the focus from feature engineering to feature learning. However, the black-box nature of deep learning makes it hard to understand the type of information learned by the network and how it relates to handcrafted features. Additionally, due to the high variabil…
EEG-EMG FAConformer: Frequency Aware Conv-Transformer for the fusion of EEG and EMG
College of Electronic；Information Engineering Tongji University Shanghai, China 0009-0008-4147-3310；School of Software Engineering Tongji University Shanghai, China；Information Engineering Tongji University Shanghai, China；Department of Computer Science；Engineering Shanghai Jiao Tong University Shanghai, China
Recent human computer-interaction (HCI) studies using electromyography (EMG) and inertial measurement units (IMUs) for upper-limb gesture recognition have claimed that inertial measurements alone result in higher classification accuracy than EMG. In biomedical research such as in prosthesis control, however, EMG remains the gold standard for providing gesture specific information, exceeding the performance of IMUs alone. This study, therefore, presents a preliminary investigation of these conflicting claims between these converging research fields. Previous claims from both fields were verifie…
Recently, surface electromyogram (EMG) has been proposed as a novel biometric trait for addressing some key limitations of current biometrics, such as spoofing and liveness. The EMG signals possess a unique characteristic: they are inherently different for individuals (biometrics), and they can be customized to realize multi-length codes or passwords (for example, by performing different gestures). However, current EMG-based biometric research has two critical limitations: 1) a small subject pool, compared to other more established biometric traits, and 2) limited to single-session or single-d…
Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encoder
Khoury College of Computer Sciences, Northeastern University（东北大学卡内基计算机科学学院）；Department of Electrical and Computer Engineering, Northeastern University（东北大学电气与计算机工程系）；Department of Physical Therapy, Movement, and Rehabilitation Sciences, Northeastern University（东北大学物理治疗、运动与康复科学系）
This work presents an innovative application of the well-known concept of cortico-muscular coherence for the classification of various motor tasks, i.e., grasps of different kinds of objects. Our approach can classify objects with different weights (motor-related features) and different surface frictions (haptics-related features) with high accuracy (over 0:8). The outcomes presented here provide information about the synchronization existing between the brain and the muscles during specific activities; thus, this may represent a new effective way to perform activity recognition.
Extraction of the movement-related high-gamma (80 - 160 Hz) in electroencephalogram (EEG) from traumatic brain injury (TBI) patients who have had hemicraniectomies, remains challenging due to a confounding bandwidth overlap with surface electromyogram (EMG) artifacts related to facial and head movements. In part 1, we described an augmented independent component analysis (ICA) approach for removal of EMG artifacts from EEG, and referred to as EMG Reduction by Adding Sources of EMG (ERASE). Here, we tested ERASE on EEG recorded from six TBI patients with hemicraniectomies while they performed a…
Based on recent health statistics, there are several thousands of people with limb disability and gait disorders that require a medical assistance. A robot assisted rehabilitation therapy can help them recover and return to a normal life. In this scenario, a successful methodology is to use the EMG signal based information to control the support robotics. For this mechanism to function properly, the EMG signal from the muscles has to be sensed and then the biological motor intention has to be decoded and finally the resulting information has to be communicated to the controller of the robot. A…
Event-Driven Implementation of a Physical Reservoir Computing Framework for superficial EMG-based Gesture Recognition
James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；Institute of Neuroinformatics, University of Zurich and ETH Zurich（苏黎世大学神经信息研究所）；University of Glasgow（格拉斯哥大学）；School of Science and Engineering, University of Dundee（邓迪大学科学与工程学院）
Electromyography signals can be used as training data by machine learning models to classify various gestures. We seek to produce a model that can classify six different hand gestures with a limited number of samples that generalizes well to a wider audience while comparing the effect of our feature extraction results on model accuracy to other more conventional methods such as the use of AR parameters on a sliding window across the channels of a signal. We appeal to a set of more elementary methods such as the use of random bounds on a signal, but desire to show the power these methods can ca…
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Colorado Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Electromyography is a promising approach to the gesture recognition of humans if an efficient classifier with high accuracy is available. In this paper, we propose to utilize Extreme Value Machine (EVM) as a high-performance algorithm for the classification of EMG signals. We employ reflection coefficients obtained from an Autoregressive (AR) model to train a set of classifiers. Our experimental results indicate that EVM has better accuracy in comparison to the conventional classifiers approved in the literature based on K-Nearest Neighbors (KNN) and Support Vector Machine (SVM).
The purpose of this document is to help individuals use the "Essential Motor Cortex Signal Processing MATLAB Toolbox". The toolbox implements various methods for three major aspects of investigating human motor cortex from Neuroscience view point: (1) ERP estimation and quantification, (2) Cortical Functional Connectivity analysis and (3) EMG quantification. The toolbox -- which is distributed under the terms of the GNU GENERAL PUBLIC LICENSE as a set of MATLAB R routines -- can be downloaded directly at the address: http://oset.ir/category.php?dir=Tools or from the public repository on GitHub…
Electromyography (EMG) signal analysis is a popular method for controlling prosthetic and gesture control equipment. For portable systems, such as prosthetic limbs, real-time low-power operation on embedded processors is critical, but to date, there has been no record of how existing EMG analysis approaches support such deployments. This paper presents a novel approach to time-domain classification of multi-channel EMG signals harnessed from randomly-placed sensors according to the wrist-hand movements which caused their occurrence. It shows how, by employing a very small set of time-domain fe…
In this study, the four joint time-frequency (TF) moments; mean, variance, skewness, and kurtosis of TF matrix obtained from Multivariate Synchrosqueezing Transform (MSST) are proposed as features for hand gesture recognition. A publicly available dataset containing surface EMG (sEMG) signals of 40 subjects performing 10 hand gestures, was used. The distinguishing power of the feature variables for the tested gestures was evaluated according to their p values obtained from the Kruskal-Wallis (KW) test. It is concluded that the mean, variance, skewness, and kurtosis of TF matrices can be candid…
University of Padova（帕多瓦大学）；National Center of Neurology and Psychiatry（国立精神·神经医疗研究中心）
Internet of Things is rapidly spreading across several fields, including healthcare, posing relevant questions related to communication capabilities, energy efficiency and sensors unobtrusiveness. Particularly, in the context of recognition of gestures, e.g., grasping of different objects, brain and muscular activity could be simultaneously recorded via EEG and EMG, respectively, and analyzed to identify the gesture that is being accomplished, and the quality of its performance. This paper proposes a new algorithm that aims (i) to robustly extract the most relevant features to classify differe…
Objective: Surface electromyogram (EMG) signals have typically been assumed to follow a Gaussian distribution. However, the presence of non-Gaussian signals associated with muscle activity has been reported in recent studies, and there is no general model of the distribution of EMG signals that can explain both non-Gaussian and Gaussian distributions within a unified scheme. Methods: In this paper, we describe the formulation of a non-Gaussian EMG model based on a scale mixture distribution. In the model, an EMG signal at a certain time follows a Gaussian distribution, and its variance is hand…
Lahore University of Management Sciences (LUMS)（拉合尔管理科学大学）；Hazara University（哈扎拉大学）；Islamic University（伊斯兰大学）
Electromyography (EMG) signals have been successfully employed for driving prosthetic limbs of a single or double degree of freedom. This principle works by using the amplitude of the EMG signals to decide between one or two simpler movements. This method underperforms as compare to the contemporary advances done at the mechanical, electronics, and robotics end, and it lacks intuition. Recently, research on myoelectric control based on pattern recognition (PR) shows promising results with the aid of machine learning classifiers. Using the approach termed as, EMG-PR, EMG signals are divided int…
University Clinic Tübingen（蒂宾根大学医院）；International Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统国际研究学院）
State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional mapping between forearm EMG activity and hand kinematics is only known with limited accuracy. Therefore, no robust method exists for the reliable computation of control signals for the independent and proportional actuation of more than two DOFs. A common approach to de…
Surface electromyography (sEMG) provides an intuitive and non-invasive interface from which to control machines. However, preserving the myoelectric control system's performance over multiple days is challenging, due to the transient nature of the signals obtained with this recording technique. In practice, if the system is to remain usable, a time-consuming and periodic recalibration is necessary. In the case where the sEMG interface is employed every few days, the user might need to do this recalibration before every use. Thus, severely limiting the practicality of such a control method. Con…
CERN（欧洲核子研究中心）；German Aerospace Center (DLR)（德国航空航天中心）；Department of Industrial Engineering, DII, University of Trento（特伦托大学工业工程系）
This work has been conducted in the context of pattern-recognition-based control for electromyographic prostheses. It presents a k-nearest neighbour (kNN) classification technique for gesture recognition, extended by a proportionality scheme. The methods proposed are practically implemented and validated. Datasets are captured by means of a state-of-the-art 8-channel electromyography (EMG) armband positioned on the forearm. Based on this data, the influence of kNN's parameters is analyzed in pilot experiments. Moreover, the effect of proportionality scaling and rest thresholding schemes is inv…
The performance of upper-limb prostheses is currently limited by the relatively poor functionality of unintuitive control schemes. This paper proposes to extract, from multichannel electromyographic signals (EMG), motor neuron spike trains and project them into lower dimensional continuous signals, which are used as multichannel proportional inputs to control the prosthetic's actuators. These control signals are an estimation of the common synaptic input that the motor neurons receive. We use the simplest of metric learning approaches known as principal component analysis (PCA), as a linear un…
Developing electroencephalogram (EEG) based brain-computer interface (BCI) systems is challenging. In this study, we analyzed natural grasp actions from EEG. Ten healthy subjects participated in this experiment. They executed and imagined three sustained grasp actions. We proposed a novel approach which estimates muscle activity patterns from EEG signals to improve the overall classification accuracy. For implementation, we have recorded EEG and electromyogram (EMG) simultaneously. Using the similarity of the estimated pattern from EEG signals compare to the activity pattern from EMG signals s…
