Deep Learning in EEG: Advance of the Last Ten-Year Critical Period
Sichuan University（四川大学）；Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）；Polish Academy of Sciences（波兰科学院）；University of Essex（埃塞克斯大学）；Wuyi University（五邑大学）
Shu Gong, Kaibo Xing, Andrzej Cichocki, Junhua Li
备注：
Accepted for publication in the IEEE Transactions on Cognitive and Developmental Systems
发表信息：
IEEE Transactions on Cognitive and Developmental Systems, 2021
Deep learning has achieved excellent performance in a wide range of domains, especially in speech recognition and computer vision. Relatively less work has been done for EEG, but there is still significant progress attained in the last decade. Due to the lack of a comprehensive and topic widely covered survey for deep learning in EEG, we attempt to summarize recent progress to provide an overview, as well as perspectives for future developments. We first briefly mention the artifacts removal for EEG signal and then introduce deep learning models that have been utilized in EEG processing and classification. Subsequently, the applications of deep learning in EEG are reviewed by categorizing them into groups such as brain-computer interface, disease detection, and emotion recognition. They are followed by the discussion, in which the pros and cons of deep learning are presented and future directions and challenges for deep learning in EEG are proposed. We hope that this paper could serve as a summary of past work for deep learning in EEG and the beginning of further developments and achievements of EEG studies based on deep learning.
Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；ENS Paris-Saclay（巴黎萨克雷高等师范学校）；Centre Borelli（博雷利中心）；Université Sorbonne Paris Nord（索邦巴黎北大学）；L2TI（L2TI实验室）；eXascale Infolab；University of Fribourg（弗里堡大学）
Recently, there has been growing interest in the analysis of spectrograms of ElectroEncephaloGram (EEG), particularly to study the neural correlates of (un)-consciousness during General Anesthesia (GA). Indeed, it has been shown that order three tensors (channels x frequencies x times) are a natural and useful representation of these signals. However this encoding entails significant difficulties, especially for convolutional sparse coding (CSC) as existing methods do not take advantage of the particularities of tensor representation, such as rank structures, and are vulnerable to the high lev…
Deep learning (DL) has been widely investigated in a vast majority of applications in electroencephalography (EEG)-based brain-computer interfaces (BCIs), especially for motor imagery (MI) classification in the past five years. The mainstream DL methodology for the MI-EEG classification exploits the temporospatial patterns of EEG signals using convolutional neural networks (CNNs), which have remarkably succeeded in visual images. However, since the statistical characteristics of visual images depart radically from EEG signals, a natural question arises whether an alternative network architectu…
A new deep learning-based electroencephalography (EEG) signal analysis framework is proposed. While deep neural networks, specifically convolutional neural networks (CNNs), have gained remarkable attention recently, they still suffer from high dimensionality of the training data. Two-dimensional input images of CNNs are more vulnerable to be redundant versus one-dimensional input time-series of conventional neural networks. In this study, we propose a new dimensionality reduction framework for reducing the dimension of CNN inputs based on the tensor decomposition of the time-frequency represen…
The brain computer interface (BCI) systems are utilized for transferring information among humans and computers by analyzing electroencephalogram (EEG) recordings.The process of mentally previewing a motor movement without generating the corporal output can be described as motor imagery (MI).In this emerging research field, the number of commands is also limited in relation to the number of MI tasks; in the current literature, mostly two or four commands (classes) are studied. As a solution to this problem, it is recommended to use mental tasks as well as MI tasks. Unfortunately, the use of th…
Objective: The use of deep learning for electroencephalography (EEG) classification tasks has been rapidly growing in the last years, yet its application has been limited by the relatively small size of EEG datasets. Data augmentation, which consists in artificially increasing the size of the dataset during training, can be employed to alleviate this problem. While a few augmentation transformations for EEG data have been proposed in the literature, their positive impact on performance is often evaluated on a single dataset and compared to one or two competing augmentation methods. This work p…
One of the current issues in Brain-Computer Interface is how to deal with noisy Electroencephalography measurements organized as multidimensional datasets. On the other hand, recently, significant advances have been made in multidimensional signal completion algorithms that exploit tensor decomposition models to capture the intricate relationship among entries in a multidimensional signal. We propose to use tensor completion applied to EEG data for improving the classification performance in a motor imagery BCI system with corrupted measurements. Noisy measurements are considered as unknowns t…
Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））；Zhengzhou University of Light Industry（郑州轻工业大学）；Nanjing University（南京大学）；Aalto University（阿尔托大学）；Beijing Institute of Technology（北京理工大学）
Emotion recognition technology through analyzing the EEG signal is currently an essential concept in Artificial Intelligence and holds great potential in emotional health care, human-computer interaction, multimedia content recommendation, etc. Though there have been several works devoted to reviewing EEG-based emotion recognition, the content of these reviews needs to be updated. In addition, those works are either fragmented in content or only focus on specific techniques adopted in this area but neglect the holistic perspective of the entire technical routes. Hence, in this paper, we review…
EEG Foundation Models: A Critical Review of Current Progress and Future Directions
Department of Computer Science & Engineering, University of Minnesota Twin Cities（计算机科学与工程系，明尼苏达大学双城分校）；Department of Bioengineering, University of Illinois at Urbana-Champaign（生物工程系，伊利诺伊大学厄巴纳-香槟分校）；Department of Neurology, Mayo Clinic（神经病学系，梅奥诊所）；Department of Neurology, University of Minnesota Twin Cities（神经病学系，明尼苏达大学双城分校）
Huazhong University of Science and Technology（华中科技大学）；Shanghai Jiao Tong University（上海交通大学）
A brain-computer interface (BCI) enables a user to communicate with a computer directly using brain signals. The most common non-invasive BCI modality, electroencephalogram (EEG), is sensitive to noise/artifact and suffers between-subject/within-subject non-stationarity. Therefore, it is difficult to build a generic pattern recognition model in an EEG-based BCI system that is optimal for different subjects, during different sessions, for different devices and tasks. Usually, a calibration session is needed to collect some training data for a new subject, which is time-consuming and user unfrie…
University of Rome Tor Vergata（罗马第二大学）；Drexel University（德雷塞尔大学）；University of Pisa（比萨大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
The brain computer interface (BCI) is a nonstimulatory direct and occasionally bidirectional communication link between the brain and a computer or an external device. Classically, EEG-based BCI algorithms have relied on models such as support vector machines and linear discriminant analysis or multiclass common spatial patterns. During the last decade, however, more sophisticated machine learning architectures, such as convolutional neural networks, recurrent neural networks, long short-term memory networks and gated recurrent unit networks, have been extensively used to enhance discriminabil…
St. Joseph’s College of Engineering（圣约瑟夫工程学院）；Texas A&M University-Commerce（德州农工大学Commerce分校）；Beirut Arab University（贝鲁特阿拉伯大学）；Anna University（安娜大学）
This paper evaluates the approach of imaging timeseries data such as EEG in the diagnosis of epilepsy through Deep Neural Network (DNN). EEG signal is transformed into an RGB image using Gramian Angular Summation Field (GASF). Many such EEG epochs are transformed into GASF images for the normal and focal EEG signals. Then, some of the widely used Deep Neural Networks for image classification problems are used here to detect the focal GASF images. Three pre-trained DNN such as the AlexNet, VGG16, and VGG19 are validated for epilepsy detection based on the transfer learning approach. Furthermore…
Since last 2 decades, High Frequency Oscillations (HFOs) are studied as a promising biomarker to localize the epileptogenic zone of patients with refractory focal epilepsy. As HFOs visual detection is time consuming and subjective, automatization of HFO detection is required. Most HFO detectors were developed on invasive electroencephalograms (iEEG) whereas scalp electroencephalograms (EEG) are used in clinical routine. In order HFO detection can benefit to more patients, scalp HFO detectors has to be developed. However, HFOs identification in scalp EEG is more challenging than in iEEG since s…
Riemannian Geometry-Based EEG Approaches: A Literature Review
Sapienza University of Rome（罗马第一大学）；University of Biskra（比斯克拉大学）；University of Djelfa（杰勒法大学）；Polytechnic Military School（军事理工学院）；Italian National Research Council（意大利国家研究委员会）；Czestochowa University of Technology（琴斯托霍瓦理工大学）
In this paper, the deep learning (DL) approach is applied to a joint training scheme for asynchronous motor imagery-based Brain-Computer Interface (BCI). The proposed DL approach is a cascade of one-dimensional convolutional neural networks and fully-connected neural networks (CNN-FC). The focus is mainly on three types of brain responses: non-imagery EEG (\textit{background EEG}), (\textit{pure imagery}) EEG, and EEG during the transitional period between background EEG and pure imagery (\textit{transitional imagery}). The study of transitional imagery signals should provide greater insight i…
Indian Institute of Information Technology Allahabad（印度阿拉哈巴德信息技术学院）
Emotion recognition using EEG signals is an emerging area of research due to its broad applicability in BCI. Emotional feelings are hard to stimulate in the lab. Emotions do not last long, yet they need enough context to be perceived and felt. However, most EEG-related emotion databases either suffer from emotionally irrelevant details (due to prolonged duration stimulus) or have minimal context doubting the feeling of any emotion using the stimulus. We tried to reduce the impact of this trade-off by designing an experiment in which participants are free to report their emotional feelings simu…
Low-Latency Neural Inference on an Edge Device for Real-Time Handwriting Recognition from EEG Signals
Electrical and Computer Engineering, University of Florida（电子与计算机工程系，佛罗里达大学）；Computer and Information Science and Engineering, University of Florida（计算机与信息科学与工程，佛罗里达大学）；Behavioral and Cognitive Neuroscience, University of Florida（行为与认知神经科学，佛罗里达大学）
Non-invasive brain-computer interface technology has been developed for detecting human mental states with high performances. Detection of the pilots' mental states is particularly critical because their abnormal mental states could cause catastrophic accidents. In this study, we presented the feasibility of classifying distraction levels (namely, normal state, low distraction, and high distraction) by applying the deep learning method. To the best of our knowledge, this study is the first attempt to classify distraction levels under a flight environment. We proposed a model for classifying di…
Faubert Lab（福贝尔实验室）；Université de Montréal（蒙特利尔大学）；Inria（法国国家信息与自动化研究所）；Université Paris-Saclay（巴黎萨克雷大学）；InteraXon Inc.（因特拉克森公司）；MuSAE Lab（穆赛实验室）；INRS-EMT（加拿大国家科学研究院能源、材料与电信研究所）；Université du Québec（魁北克大学）
Electroencephalography (EEG) is a complex signal and can require several years of training to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. In this work, we review 156 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain-computer …
The large range of potential applications, not only for patients but also for healthy people, that could be achieved by affective BCI (aBCI) makes more latent the necessity of finding a commonly accepted protocol for real-time EEG-based emotion recognition. Based on wavelet package for spectral feature extraction, attending to the nature of the EEG signal, we have specified some of the main parameters needed for the implementation of robust positive and negative emotion classification. 12 seconds has resulted as the most appropriate sliding window size; from that, a set of 20 target frequency-…
Korea Atomic Energy Research Institute（韩国原子能研究院）；Gwangju Institute of Science and Technology (GIST)（光州科学技术院）
A deep learning model, named IITNet, is proposed to learn intra- and inter-epoch temporal contexts from raw single-channel EEG for automatic sleep scoring. To classify the sleep stage from half-minute EEG, called an epoch, sleep experts investigate sleep-related events and consider the transition rules between the found events. Similarly, IITNet extracts representative features at a sub-epoch level by a residual neural network and captures intra- and inter-epoch temporal contexts from the sequence of the features via bidirectional LSTM. The performance was investigated for three datasets as th…
Brain-computer interface (BCI) technologies have been widely used in many areas. In particular, non-invasive technologies such as electroencephalography (EEG) or near-infrared spectroscopy (NIRS) have been used to detect motor imagery, disease, or mental state. It has been already shown in literature that the hybrid of EEG and NIRS has better results than their respective individual signals. The fusion algorithm for EEG and NIRS sources is the key to implement them in real-life applications. In this research, we propose three fusion methods for the hybrid of the EEG and NIRS-based brain-comput…
