Tensor-CSPNet: A Novel Geometric Deep Learning Framework for Motor Imagery Classification
Nanyang Technological University（南洋理工大学）
Ce Ju, Cuntai Guan
备注：
15 pages, 10 figures, 12 tables; This work has been accepted by the IEEE Transactions on Neural Networks and Learning Systems. Copyright will be transferred without notice, after which this version may no longer be accessible
Deep learning (DL) has been widely investigated in a vast majority of applications in electroencephalography (EEG)-based brain-computer interfaces (BCIs), especially for motor imagery (MI) classification in the past five years. The mainstream DL methodology for the MI-EEG classification exploits the temporospatial patterns of EEG signals using convolutional neural networks (CNNs), which have remarkably succeeded in visual images. However, since the statistical characteristics of visual images depart radically from EEG signals, a natural question arises whether an alternative network architecture exists apart from CNNs. To address this question, we propose a novel geometric deep learning (GDL) framework called Tensor-CSPNet, which characterizes spatial covariance matrices derived from EEG signals on symmetric positive definite (SPD) manifolds and fully captures the temporospatiofrequency patterns using existing deep neural networks on SPD manifolds, integrating with experiences from many successful MI-EEG classifiers to optimize the framework. In the experiments, Tensor-CSPNet attains or slightly outperforms the current state-of-the-art performance on the cross-validation and holdout scenarios in two commonly-used MI-EEG datasets. Moreover, the visualization and interpretability analyses also exhibit the validity of Tensor-CSPNet for the MI-EEG classification. To conclude, in this study, we provide a feasible answer to the question by generalizing the DL methodologies on SPD manifolds, which indicates the start of a specific GDL methodology for the MI-EEG classification.
A trained T1 class Convolutional Neural Network (CNN) model will be used to examine its ability to successfully identify motor imagery when fed pre-processed electroencephalography (EEG) data. In theory, and if the model has been trained accurately, it should be able to identify a class and label it accordingly. The CNN model will then be restored and used to try and identify the same class of motor imagery data using much smaller sampled data in an attempt to simulate live data.
Zhejiang University（浙江大学）；Maebashi Institute of Technology（前桥工业大学）；Monash University（莫纳什大学）；Ningbo Research Institute, Zhejiang University（浙江大学宁波研究院）
Motor imagery classification is of great significance to humans with mobility impairments, and how to extract and utilize the effective features from motor imagery electroencephalogram(EEG) channels has always been the focus of attention. There are many different methods for the motor imagery classification, but the limited understanding on human brain requires more effective methods for extracting the features of EEG data. Graph neural networks(GNNs) have demonstrated its effectiveness in classifying graph structures; and the use of GNN provides new possibilities for brain structure connectio…
We introduce here the idea of Meta-Learning for training EEG BCI decoders. Meta-Learning is a way of training machine learning systems so they learn to learn. We apply here meta-learning to a simple Deep Learning BCI architecture and compare it to transfer learning on the same architecture. Our Meta-learning strategy operates by finding optimal parameters for the BCI decoder so that it can quickly generalise between different users and recording sessions -- thereby also generalising to new users or new sessions quickly. We tested our algorithm on the Physionet EEG motor imagery dataset. Our ap…
Robust Feature Engineering Techniques for Designing Efficient Motor Imagery-Based BCI-Systems
Salim Habib University（萨利姆哈比布大学）；NUST（国家科技大学）；Riphah International University（里法国际大学）；NED University（NED大学）；Institute of Business Administration（商业管理学院）
Dealing with irregular domains, graph signal processing (GSP) has attracted much attention especially in brain imaging analysis. Motor imagery tasks are extensively utilized in brain-computer interface (BCI) systems that perform classification using features extracted from Electroencephalogram signals. In this paper, a GSP-based approach is presented for two-class motor imagery tasks classification. The proposed method exploits simultaneous diagonalization of two matrices that quantify the covariance structure of graph spectral representation of data from each class, providing a discriminative…
Similar to most of the real world data, the ubiquitous presence of non-stationarities in the EEG signals significantly perturb the feature distribution thus deteriorating the performance of Brain Computer Interface. In this letter, a novel method is proposed based on Joint Approximate Diagonalization (JAD) to optimize stationarity for multiclass motor imagery Brain Computer Interface (BCI) in an information theoretic framework. Specifically, in the proposed method, we estimate the subspace which optimizes the discriminability between the classes and simultaneously preserve stationarity within …
Northeast Electric Power University（东北电力大学）；Changchun University of Science and Technology（长春理工大学）；University of Sydney（悉尼大学）
Recognition accuracy and response time are both critically essential ahead of building practical electroencephalography (EEG) based brain-computer interface (BCI). Recent approaches, however, have either compromised in the classification accuracy or responding time. This paper presents a novel deep learning approach designed towards remarkably accurate and responsive motor imagery (MI) recognition based on scalp EEG. Bidirectional Long Short-term Memory (BiLSTM) with the Attention mechanism manages to derive relevant features from raw EEG signals. The connected graph convolutional neural netwo…
Objective. Many electroencephalogram (EEG)-based brain-computer interface (BCI) systems use a large amount of channels for higher performance, which is time-consuming to set up and inconvenient for practical applications. Finding an optimal subset of channels without compromising the performance is a necessary and challenging task. Approach. In this article, we proposed a cross-correlation based discriminant criterion (XCDC) which assesses the importance of a channel for discriminating the mental states of different motor imagery (MI) tasks. Channels are ranked and selected according to the pr…
Brain-computer interface (BCI) decodes brain signals to understand user intention and status. Because of its simple and safe data acquisition process, electroencephalogram (EEG) is commonly used in non-invasive BCI. One of EEG paradigms, motor imagery (MI) is commonly used for recovery or rehabilitation of motor functions due to its signal origin. However, the EEG signals are an oscillatory and non-stationary signal that makes it difficult to collect and classify MI accurately. In this study, we proposed a band-power feature refining convolutional neural network (BFR-CNN) which is composed of …
In this paper we propose a new pre-processing technique of Electroencephalography (EEG) signals produced by motor imagery movements. This technique results to an accelerated determination of the imagery movement and the command to carry it out, within the time limits imposed by the requirements of brain-based real-time control of rehabilitation devices, making thus feasible to drive these devices according to patient's will. Based on event related desynchronization and synchronization (ERD/ERS) of motor imagery, the received patient signal is first subjected to the removal of environmental, sy…
A brain-computer interface (BCI) provides a direct communication pathway between user and external devices. Electroencephalogram (EEG) motor imagery (MI) paradigm is widely used in non-invasive BCI to obtain encoded signals contained user intention of movement execution. However, EEG has intricate and non-stationary properties resulting in insufficient decoding performance. By imagining numerous movements of a single-arm, decoding performance can be improved without artificial command matching. In this study, we collected intuitive EEG data contained the nine different types of movements of a …
Motor-imagery based brain-computer interfaces (MI-BCI) have the potential to become ground-breaking technologies for neurorehabilitation, the reestablishment of non-muscular communication and commands for patients suffering from neuronal disorders and disabilities, but also outside of clinical practice, for video game control and other entertainment purposes. However, due to the noisy nature of the used EEG signal, reliable BCI systems require specialized procedures for features optimization and extraction. This paper compares the two approaches, the Common Spatial Patterns with Linear Discrim…
Brain-Computer Interfaces (BCI) based on motor imagery translate mental motor images recognized from the electroencephalogram (EEG) to control commands. EEG patterns of different imagination tasks, e.g. hand and foot movements, are effectively classified with machine learning techniques using band power features. Recently, also Convolutional Neural Networks (CNNs) that learn both effective features and classifiers simultaneously from raw EEG data have been applied. However, CNNs have two major drawbacks: (i) they have a very large number of parameters, which thus requires a very large number o…
In this paper, the deep learning (DL) approach is applied to a joint training scheme for asynchronous motor imagery-based Brain-Computer Interface (BCI). The proposed DL approach is a cascade of one-dimensional convolutional neural networks and fully-connected neural networks (CNN-FC). The focus is mainly on three types of brain responses: non-imagery EEG (\textit{background EEG}), (\textit{pure imagery}) EEG, and EEG during the transitional period between background EEG and pure imagery (\textit{transitional imagery}). The study of transitional imagery signals should provide greater insight i…
Queen Mary University of London（伦敦玛丽女王大学）；Nanyang Technological University（南洋理工大学）；Suranaree University of Technology（苏拉那里理工大学）
Advances in the motor imagery (MI)-based brain-computer interfaces (BCIs) allow control of several applications by decoding neurophysiological phenomena, which are usually recorded by electroencephalography (EEG) using a non-invasive technique. Despite great advances in MI-based BCI, EEG rhythms are specific to a subject and various changes over time. These issues point to significant challenges to enhance the classification performance, especially in a subject-independent manner. To overcome these challenges, we propose MIN2Net, a novel end-to-end multi-task learning to tackle this task. We i…
Common spatial pattern (CSP) is a popular feature extraction method for electroencephalogram (EEG) motor imagery (MI). This study modifies the conventional CSP algorithm to improve the multi-class MI classification accuracy and ensure the computation process is efficient. The EEG MI data is gathered from the Brain-Computer Interface (BCI) Competition IV. At first, a bandpass filter and a time-frequency analysis are performed for each experiment trial. Then, the optimal EEG signals for every experiment trials are selected based on the signal energy for CSP feature extraction. In the end, the ex…
Brain-computer interface (BCI) systems are usually designed specifically for each subject based on motor imagery. Therefore, the usability of these networks has become a significant challenge. The network has to be designed separately for each user, which is time-consuming for the user. Therefore, this study proposes a method by which the calibration time is significantly reduced while the classification accuracy is increased. In this method, we calibrated the features extracted from the motor imagery task by dividing the features extracted from the resting-state into both open-eye and closed-…
ETH Zürich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）
This paper presents an accurate and robust embedded motor-imagery brain-computer interface (MI-BCI). The proposed novel model, based on EEGNet, matches the requirements of memory footprint and computational resources of low-power microcontroller units (MCUs), such as the ARM Cortex-M family. Furthermore, the paper presents a set of methods, including temporal downsampling, channel selection, and narrowing of the classification window, to further scale down the model to relax memory requirements with negligible accuracy degradation. Experimental results on the Physionet EEG Motor Movement/Image…
Detecting the salient parts of motor-imagery electroencephalogram (MI-EEG) signals can enhance the performance of the brain-computer interface (BCI) system and reduce the computational burden required for processing lengthy MI-EEG signals. In this paper, we propose an unsupervised method based on the self-attention mechanism to detect the salient intervals of MI-EEG signals automatically. Our suggested method can be used as a preprocessing step within any BCI algorithm to enhance its performance. The effectiveness of the suggested method is evaluated on the most widely used BCI algorithm, the …
West China Hospital, Sichuan University（四川大学华西医院）；Southwest Jiaotong University（西南交通大学）
The electroencephalography (EEG) signal is a non-stationary, stochastic, and highly non-linear bioelectric signal for which achieving high classification accuracy is challenging, especially when the number of subjects is limited. As frequently used solution, classifiers based on multilayer neural networks has to be implemented without large training data sets and careful tuning. This paper proposes a manifold learning framework to classify two types of EEG data from motor imagery (MI) tasks by discovering lower dimensional geometric structures. For feature extraction, it is implemented by Comm…
This paper deals with the detection of mu-suppression from electroencephalographic (EEG) signals in brain-computer interface (BCI). For this purpose, an efficient algorithm is proposed based on a statistical model and a linear classifier. Precisely, the generalized extreme value distribution (GEV) is proposed to represent the power spectrum density of the EEG signal in the central motor cortex. The associated three parameters are estimated using the maximum likelihood method. Based on these parameters, a simple and efficient linear classifier was designed to classify three types of events: ima…
Brain-computer interface (BCI) systems have potential as assistive technologies for individuals with severe motor impairments. Nevertheless, individuals must first participate in many training sessions to obtain adequate data for optimizing the classification algorithm and subsequently acquiring brain-based control. Such traditional training paradigms have been dubbed unengaging and unmotivating for users. In recent years, it has been shown that the synergy of virtual reality (VR) and a BCI can lead to increased user engagement. This study created a 3-class BCI with a rather elaborate EEG sign…
Nowadays, Brain Computer Interface has an important role in the life quality of parallelized people. However, this technique is mainly affected by the quality of the recorded signal in each trial. This problem could be solved by rejecting low-quality trials. But developing the processing based on the recorded signal from the brain, which is a mixture of the target signal plus noise and artifact, would not be favorable in situations that all trials have low quality. This paper solves this problem by presenting a new fast algorithm for separating recorded source signals. Results indicate the imp…
Virginia Polytechnic Institute and State University（弗吉尼亚理工大学）；Handong Global University（韩东国际大学）
Classification of EEG-based motor imagery (MI) is a crucial non-invasive application in brain-computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network, which showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does…
ETH Zurich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）；IBM Research-Zurich（IBM苏黎世研究院）
Successful motor-imagery brain-computer interface (MI-BCI) algorithms either extract a large number of handcrafted features and train a classifier, or combine feature extraction and classification within deep convolutional neural networks (CNNs). Both approaches typically result in a set of real-valued weights, that pose challenges when targeting real-time execution on tightly resource-constrained devices. We propose methods for each of these approaches that allow transforming real-valued weights to binary numbers for efficient inference. Our first method, based on sparse bipolar random projec…
In a self-paced motor-imagery brain-computer interface (MI-BCI), the onsets of the MI commands presented in a continuous electroencephalogram (EEG) signal are unknown. To detect these onsets, most self-paced approaches apply a window function on the continuous EEG signal and split it into long segments for further analysis. As a result, the system has a high latency. To reduce the system latency, we propose an algorithm based on the time series prediction concept and use the data of the previously received time samples to predict the upcoming time samples. Our predictor is an encoder-decoder (…
ETH Zürich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）；Huawei Technologies（华为技术有限公司）
Motor-Imagery Brain--Machine Interfaces (MI-BMIs)promise direct and accessible communication between human brains and machines by analyzing brain activities recorded with Electroencephalography (EEG). Latency, reliability, and privacy constraints make it unsuitable to offload the computation to the cloud. Practical use cases demand a wearable, battery-operated device with low average power consumption for long-term use. Recently, sophisticated algorithms, in particular deep learning models, have emerged for classifying EEG signals. While reaching outstanding accuracy, these models often exceed…
Northeast Electric Power University（东北电力大学）；City University of Hong Kong（香港城市大学）；Jinan Vocational College of Nursing（济南护理职业学院）；University of Sydney（悉尼大学）
Towards developing effective and efficient brain-computer interface (BCI) systems, precise decoding of brain activity measured by electroencephalogram (EEG), is highly demanded. Traditional works classify EEG signals without considering the topological relationship among electrodes. However, neuroscience research has increasingly emphasized network patterns of brain dynamics. Thus, the Euclidean structure of electrodes might not adequately reflect the interaction between signals. To fill the gap, a novel deep learning framework based on the graph convolutional neural networks (GCNs) is present…
University of Padova（帕多瓦大学）；National Centre for Neurology and Psychiatry（国家神经与精神病学中心）；National Inter-University Consortium for Telecommunications (CNIT)（全国大学间电信联盟（CNIT））；University of Coimbra（科英布拉大学）；Polytechnic Institute of Tomar（托马尔理工学院）
Motor imagery (MI)-based brain-computer interface (BCI) systems are being increasingly employed to provide alternative means of communication and control for people suffering from neuro-motor impairments, with a special effort to bring these systems out of the controlled lab environments. Hence, accurately classifying MI from brain signals, e.g., from electroencephalography (EEG), is essential to obtain reliable BCI systems. However, MI classification is still a challenging task, because the signals are characterized by poor SNR, high intra-subject and cross-subject variability. Deep learning …
Developments in Brain Computer Interfaces (BCIs) are empowering those with severe physical afflictions through their use in assistive systems. Common methods of achieving this is via Motor Imagery (MI), which maps brain signals to code for certain commands. Electroencephalogram (EEG) is preferred for recording brain signal data on account of it being non-invasive. Despite their potential utility, MI-BCI systems are yet confined to research labs. A major cause for this is lack of robustness of such systems. As hypothesized by two teams during Cybathlon 2016, a particular source of the system's …
MVCNet: Multi-View Contrastive Network for Motor Imagery Classification
Ministry of Education Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（教育部图像处理与智能控制重点实验室，人工智能与自动化学院，华中科技大学）；Shenzhen Huazhong University of Science and Technology Research Institute（深圳华中科技大学研究院）
ETH Zürich（苏黎世联邦理工学院）；Nihon University（日本大学）；Huawei Technologies（华为技术有限公司）
In recent years, deep learning (DL) has contributed significantly to the improvement of motor-imagery brain-machine interfaces (MI-BMIs) based on electroencephalography(EEG). While achieving high classification accuracy, DL models have also grown in size, requiring a vast amount of memory and computational resources. This poses a major challenge to an embedded BMI solution that guarantees user privacy, reduced latency, and low power consumption by processing the data locally. In this paper, we propose EEG-TCNet, a novel temporal convolutional network (TCN) that achieves outstanding accuracy wh…
Key properties of brain-inspired hyperdimensional (HD) computing make it a prime candidate for energy-efficient and fast learning in biosignal processing. The main challenge is however to formulate embedding methods that map biosignal measures to a binary HD space. In this paper, we explore variety of such embedding methods and examine them with a challenging application of motor imagery brain-computer interface (MI-BCI) from electroencephalography (EEG) recordings. We explore embedding methods including random projections, quantization based thermometer and Gray coding, and learning HD repres…
Daegu Gyeongbuk Institute of Science and Technology (DGIST)（大邱庆北科学技术院（DGIST））
Brain-Computer Interfaces (BCI) based on Electroencephalography (EEG) signals, in particular motor imagery (MI) data have received a lot of attention and show the potential towards the design of key technologies both in healthcare and other industries. MI data is generated when a subject imagines movement of limbs and can be used to aid rehabilitation as well as in autonomous driving scenarios. Thus, classification of MI signals is vital for EEG-based BCI systems. Recently, MI EEG classification techniques using deep learning have shown improved performance over conventional techniques. Howeve…
Enhancing Motor Imagery Decoding in Brain Computer Interfaces using Riemann Tangent Space Mapping and Cross Frequency Coupling
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications（北京邮电大学信息与通信工程学院）；University of Sheffield（谢菲尔德大学）；School of Automation, Beijing University of Posts and Telecommunications（北京邮电大学自动化学院）
