EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications
University of Tasmania, Australia（塔斯马尼亚大学）；Macquarie University, Australia（麦考瑞大学）；University of Electronic Science and Technology of China, China（电子科技大学）；Huazhong University of Science and Technology, China（华中科技大学）；University of California, San Diego, USA（加利福尼亚大学圣地亚哥分校）；University of Technology Sydney, Australia（悉尼科技大学）
Brain-Computer Interface (BCI) is a powerful communication tool between users and systems, which enhances the capability of the human brain in communicating and interacting with the environment directly. Advances in neuroscience and computer science in the past decades have led to exciting developments in BCI, thereby making BCI a top interdisciplinary research area in computational neuroscience and intelligence. Recent technological advances such as wearable sensing devices, real-time data streaming, machine learning, and deep learning approaches have increased interest in electroencephalographic (EEG) based BCI for translational and healthcare applications. Many people benefit from EEG-based BCIs, which facilitate continuous monitoring of fluctuations in cognitive states under monotonous tasks in the workplace or at home. In this study, we survey the recent literature of EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensated for the gaps in the systematic summary of the past five years (2015-2019). In specific, we first review the current status of BCI and its significant obstacles. Then, we present advanced signal sensing and enhancement technologies to collect and clean EEG signals, respectively. Furthermore, we demonstrate state-of-art computational intelligence techniques, including interpretable fuzzy models, transfer learning, deep learning, and combinations, to monitor, maintain, or track human cognitive states and operating performance in prevalent applications. Finally, we deliver a couple of innovative BCI-inspired healthcare applications and discuss some future research directions in EEG-based BCIs.
Huazhong University of Science and Technology（华中科技大学）；National University of Singapore（新加坡国立大学）；Zhejiang Lab（之江实验室）
Multiple convolutional neural network (CNN) classifiers have been proposed for electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, CNN models have been found vulnerable to universal adversarial perturbations (UAPs), which are small and example-independent, yet powerful enough to degrade the performance of a CNN model, when added to a benign example. This paper proposes a novel total loss minimization (TLM) approach to generate UAPs for EEG-based BCIs. Experimental results demonstrated the effectiveness of TLM on three popular CNN classifiers for both target and non-targ…
University of Milano-Bicocca（米兰比可卡大学）；NeuroMI, Milan Center for Neuroscience（米兰神经科学中心（NeuroMI））；University of Vic-Central University of Catalonia（维克大学-加泰罗尼亚中央大学）
The ageing process may lead to cognitive and physical impairments, which may affect elderly everyday life. In recent years, the use of Brain Computer Interfaces (BCIs) based on Electroencephalography (EEG) has revealed to be particularly effective to promote and enhance rehabilitation procedures, especially by exploiting motor imagery experimental paradigms. Moreover, BCIs seem to increase patients' engagement and have proved to be reliable tools for elderly overall wellness improvement. However, EEG signals usually present a low signal-to-noise ratio and can be recorded for a limited time. Th…
Most EEG-based Brain-Computer Interfaces (BCIs) require a considerable amount of training data to calibrate the classification model, owing to the high variability in the EEG data, which manifests itself between participants, but also within participants from session to session (and, of course, from trial to trial). In general, the more complex the model, the more data for training is needed. We suggest a novel technique for augmenting the training data by generating new data from the data set at hand. Different from existing techniques, our method uses backward and forward projection using so…
Brain-Computer Interface (BCI) is an essential mechanism that interprets the human brain signal. It provides an assistive technology that enables persons with motor disabilities to communicate with the world and also empowers them to lead independent lives. The common BCI devices use Electroencephalography (EEG) electrical activity recorded from the scalp. EEG signals are noisy owing to the presence of many artifacts, namely, eye blink, head movement, and jaw movement. Such artifacts corrupt the EEG signal and make EEG analysis challenging. This issue is addressed by locating the artifacts and…
We present a mobile dataset obtained from electroencephalography (EEG) of the scalp and around the ear as well as from locomotion sensors by 24 participants moving at four different speeds while performing two brain-computer interface (BCI) tasks. The data were collected from 32-channel scalp-EEG, 14-channel ear-EEG, 4-channel electrooculography, and 9-channel inertial measurement units placed at the forehead, left ankle, and right ankle. The recording conditions were as follows: standing, slow walking, fast walking, and slight running at speeds of 0, 0.8, 1.6, and 2.0m/s, respectively. For ea…
Brain-Computer Interfaces (BCIs) enable converting the brain electrical activity of an interface user to the user commands. BCI research studies demonstrated encouraging results in different areas such as neurorehabilitation, control of artificial limbs, control of computer environments, communication and detection of diseases. Most of BCIs use scalp-electroencephalography (EEG), which is a non-invasive method to capture the brain activity. Although EEG monitoring devices are available in the market, these devices are generally lab-oriented and expensive. Day-to-day use of BCIs is impractical …
A Conversational Brain-Artificial Intelligence Interface
University of Vienna（维也纳大学）；Doctoral School Computer Science, Faculty of Computer Science, University of Vienna（维也纳大学计算机学院计算机博士学院）；Vienna Cognitive Science Hub, University of Vienna（维也纳大学维也纳认知科学中心）；Research Network Data Science, University of Vienna（维也纳大学数据科学研究网络）
Università di Pisa（比萨大学）；CAMLIN Italy s.r.l.（CAMLIN意大利有限公司）
Typical EEG-based BCI applications require the computation of complex functions over the noisy EEG channels to be carried out in an efficient way. Deep learning algorithms are capable of learning flexible nonlinear functions directly from data, and their constant processing latency is perfect for their deployment into online BCI systems. However, it is crucial for the jitter of the processing system to be as low as possible, in order to avoid unpredictable behaviour that can ruin the system's overall usability. In this paper, we present a novel encoding method, based on on deep convolutional a…
Fusing Pretrained ViTs with TCNet for Enhanced EEG Regression
University of Illinois at Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Saint Francis Preparatory High School（圣弗朗西斯预备高中）；Palo Alto High School（帕洛阿尔托高中）
Currently, strokes are the leading cause of adult disability in the United States. Traditional treatment and rehabilitation options such as physical therapy and tissue plasminogen activator are limited in their effectiveness and ability to restore mobility and function to the patient. As a result, there exists an opportunity to greatly improve the treatment for strokes. Machine learning, specifically techniques that utilize Brain-Computer Interfaces (BCIs) to help the patient either restore neurologic pathways or effectively communicate with an electronic prosthetic, show promising results whe…
Channel Reflection: Knowledge-Driven Data Augmentation for EEG-Based Brain-Computer Interfaces
Huazhong University of Science and Technology（华中科技大学）；Shenzhen Huazhong University of Science and Technology Research Institute（华中科技大学深圳研究院）；China Electronic System Technology Co., Ltd.（中国电子系统技术有限公司）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology（华中科技大学土木与水利工程学院）
Huazhong University of Science and Technology（华中科技大学）
Deep learning has made significant breakthroughs in many fields, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, deep learning models are vulnerable to adversarial attacks, in which deliberately designed small perturbations are added to the benign input samples to fool the deep learning model and degrade its performance. This paper considers transferability-based black-box attacks, where the attacker trains a substitute model to approximate the target model, and then generates adversarial examples from the substitute model to attack the target model. Learn…
Machine learning models are known to be vulnerable to adversarial perturbations in the input domain, causing incorrect predictions. Inspired by this phenomenon, we explore the feasibility of manipulating EEG-based Motor Imagery (MI) Brain Computer Interfaces (BCIs) via perturbations in sensory stimuli. Similar to adversarial examples, these \emph{adversarial stimuli} aim to exploit the limitations of the integrated brain-sensor-processing components of the BCI system in handling shifts in participants' response to changes in sensory stimuli. This paper proposes adversarial stimuli as an attack…
Riemannian Geometry-Based EEG Approaches: A Literature Review
Sapienza University of Rome（罗马第一大学）；University of Biskra（比斯克拉大学）；University of Djelfa（杰勒法大学）；Polytechnic Military School（军事理工学院）；Italian National Research Council（意大利国家研究委员会）；Czestochowa University of Technology（琴斯托霍瓦理工大学）
Huazhong University of Science and Technology（华中科技大学）；Shanghai Jiao Tong University（上海交通大学）
A brain-computer interface (BCI) enables a user to communicate with a computer directly using brain signals. The most common non-invasive BCI modality, electroencephalogram (EEG), is sensitive to noise/artifact and suffers between-subject/within-subject non-stationarity. Therefore, it is difficult to build a generic pattern recognition model in an EEG-based BCI system that is optimal for different subjects, during different sessions, for different devices and tasks. Usually, a calibration session is needed to collect some training data for a new subject, which is time-consuming and user unfrie…
Swartz Center for Computational Neuroscience（斯沃茨计算神经科学中心）；Institute of Neural Computation（神经计算研究所）；University of California, San Diego（加州大学圣地亚哥分校）
Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have shown its robustness in facilitating high-efficiency communication. State-of-the-art training-based SSVEP decoding methods such as extended Canonical Correlation Analysis (CCA) and Task-Related Component Analysis (TRCA) are the major players that elevate the efficiency of the SSVEP-based BCIs through a calibration process. However, due to notable human variability across individuals and within individuals over time, calibration (training) data collection is non-negligible and often laborious and time-consu…
The aim of this paper is to design and construct an electroencephalograph (EEG) based brain-controlled wheelchair to provide a communication bridge from the nervous system to the external technical device for people of determination or individuals suffering from partial or complete paralysis. EEG is a technique that reads the activity of the brain by capturing brain signals non-invasively using a special EEG headset. The signals acquired go through pre-processing, feature extraction and classification. This technique allows human thoughts alone to be converted to control the wheelchair. The co…
In this work, we formulate the problem of estimating and selecting task-relevant temporal signal segments from a single EEG trial in the form of a Markov decision process and propose a novel reinforcement-learning mechanism that can be combined with the existing deep-learning based BCI methods. To be specific, we devise an actor-critic network such that an agent can determine which timepoints need to be used (informative) or discarded (uninformative) in composing the intention-related features in a given trial, and thus enhancing the intention identification performance. To validate the effect…
In recent years, multiple noninvasive imaging modalities have been used to develop a better understanding of the human brain functionality, including positron emission tomography, single-photon emission computed tomography, and functional magnetic resonance imaging, all of which provide brain images with millimeter spatial resolutions. Despite good spatial resolution, time resolution of these methods are poor and values are about seconds. Electroencephalography (EEG) is a popular non-invasive electrophysiological technique of relatively very high time resolution which is used to measure electr…
EEG2MOTION: Towards Open-Vocabulary Human Motion Synthesis from Non-invasive Brain Signals
Zhejiang University（浙江大学）；Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The Hong Kong Polytechnic University（香港理工大学）；Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University（浙江大学教育部生物医学工程重点实验室）
Brain-Computer interfaces (BCIs) play a significant role in easing neuromuscular patients on controlling computers and prosthetics. Due to their high signal-to-noise ratio, steady-state visually evoked potentials (SSVEPs) has been widely used to build BCIs. However, currently developed algorithms do not predict the modulation of SSVEP amplitude, which is known to change as a function of stimulus luminance contrast. In this study, we aim to develop an integrated approach to simultaneously estimate the frequency and contrast-related amplitude modulations of the SSVEP signal. To achieve that, we …
Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directions
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China（华中科技大学人工智能与自动化学院，武汉，中国）；Zhongguancun Academy, Beijing, China（中关村学院，北京，中国）
Recent advances in deep learning have had a methodological and practical impact on brain-computer interface research. Among the various deep network architectures, convolutional neural networks have been well suited for spatio-spectral-temporal electroencephalogram signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in…
Medical Center - University of Freiburg（弗莱堡大学医学中心）；University of Freiburg（弗莱堡大学）
In this manuscript, we investigate deep invertible networks for EEG-based brain signal decoding and find them to generate realistic EEG signals as well as classify novel signals above chance. Further ideas for their regularization towards better decoding accuracies are discussed.
Effective analysis of EEG signals for potential clinical applications remains a challenging task. So far, the analysis and conditioning of EEG have largely remained sex-neutral. This paper employs a machine learning approach to explore the evidence of sex effects on EEG signals, and confirms the generality of these effects by achieving successful sex prediction of resting-state EEG signals. We have found that the brain connectivity represented by the coherence between certain sensor channels are good predictors of sex.
Drivers cognitive and physiological states affect their ability to control their vehicles. Thus, these driver states are important to the safety of automobiles. The design of advanced driver assistance systems (ADAS) or autonomous vehicles will depend on their ability to interact effectively with the driver. A deeper understanding of the driver state is, therefore, paramount. EEG is proven to be one of the most effective methods for driver state monitoring and human error detection. This paper discusses EEG-based driver state detection systems and their corresponding analysis algorithms over t…
A trained T1 class Convolutional Neural Network (CNN) model will be used to examine its ability to successfully identify motor imagery when fed pre-processed electroencephalography (EEG) data. In theory, and if the model has been trained accurately, it should be able to identify a class and label it accordingly. The CNN model will then be restored and used to try and identify the same class of motor imagery data using much smaller sampled data in an attempt to simulate live data.
Generative adversarial networks (GANs) are recently highly successful in generative applications involving images and start being applied to time series data. Here we describe EEG-GAN as a framework to generate electroencephalographic (EEG) brain signals. We introduce a modification to the improved training of Wasserstein GANs to stabilize training and investigate a range of architectural choices critical for time series generation (most notably up- and down-sampling). For evaluation we consider and compare different metrics such as Inception score, Frechet inception distance and sliced Wasser…
Despite knowing the reality of three-dimensional (3D) technology in the form of eye fatigue, this technology continues to be retained by people (especially the young community). To check what happens before and after watching a 2D and 3D movie and how this condition influences the human brain's power spectrum density (PSD), a five-member test group was arranged. In this study, electroencephalogram (EEG) was used as a neuroimaging method. EEG recordings of five individuals were taken both before and after watching 2D and 3D movies. After 2D/3D EEG recording, this record was divided into three s…
BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals
Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室）；Department of Electronic Engineering, Tsinghua University, China（清华大学电子工程系）；Department of Psychology, University of Cambridge, UK（剑桥大学心理学系）；Speech Hearing and Phonetic Sciences, University College London, UK（伦敦大学学院语音听力与语音科学系）；MRC Cognition and Brain Sciences Unit, University of Cambridge, UK（剑桥大学MRC认知与脑科学单位）
In this study, the Multivariate Empirical Mode Decomposition (MEMD) approach is applied to extract features from multi-channel EEG signals for mental state classification. MEMD is a data-adaptive analysis approach which is suitable particularly for multi-dimensional non-linear signals like EEG. Applying MEMD results in a set of oscillatory modes called intrinsic mode functions (IMFs). As the decomposition process is data-dependent, the IMFs vary in accordance with signal variation caused by functional brain activity. Among the extracted IMFs, it is found that those corresponding to high-oscill…
ETH Zürich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）；Huawei Technologies（华为技术有限公司）
Motor-Imagery Brain--Machine Interfaces (MI-BMIs)promise direct and accessible communication between human brains and machines by analyzing brain activities recorded with Electroencephalography (EEG). Latency, reliability, and privacy constraints make it unsuitable to offload the computation to the cloud. Practical use cases demand a wearable, battery-operated device with low average power consumption for long-term use. Recently, sophisticated algorithms, in particular deep learning models, have emerged for classifying EEG signals. While reaching outstanding accuracy, these models often exceed…
We present a novel solution to the problem of localization of MEG and EEG brain signals. The solution is sequential and iterative, and is based on minimizing the least-squares (LS)criterion by the Alternating Projection (AP) algorithm, which is well known in the context of array signal processing. Unlike existing scanning solutions belonging to the beamformer and multiple-signal classification (MUSIC) families, the algorithm has good performance in low signal-to-noise ratio (SNR) and can cope with closely spaced sources and any mixture of correlated sources. Results from simulated and experime…
Representation and classification of Electroencephalography (EEG) brain signals are critical processes for their analysis in cognitive tasks. Particularly, extraction of discriminative features from raw EEG signals, without any pre-processing, is a challenging task. Motivated by nuclear norm, we observed that there is a significant difference between the variances of EEG signals captured from the same brain region when a subject performs different tasks. This observation lead us to use singular value decomposition for computing dominant variances of EEG signals captured from a certain brain re…
Low-Latency Neural Inference on an Edge Device for Real-Time Handwriting Recognition from EEG Signals
Electrical and Computer Engineering, University of Florida（电子与计算机工程系，佛罗里达大学）；Computer and Information Science and Engineering, University of Florida（计算机与信息科学与工程，佛罗里达大学）；Behavioral and Cognitive Neuroscience, University of Florida（行为与认知神经科学，佛罗里达大学）
Electroencephalogram (EEG) has been one of the common neuromonitoring modalities for real-world brain-computer interfaces (BCIs) because of its non-invasiveness, low cost, and high temporal resolution. Recently, light-weight and portable EEG wearable devices based on low-density montages have increased the convenience and usability of BCI applications. However, loss of EEG decoding performance is often inevitable due to reduced number of electrodes and coverage of scalp regions of a low-density EEG montage. To address this issue, we introduce knowledge distillation (KD), a learning mechanism d…
The source localization of the human brain activities is an important resource for the recognition of cognitive state, medical disorders and a better understanding of the brain in general. In this study, we have compared 51 mother wavelets from 7 different wavelet families in a Stationary Wavelet transform (SWT) decomposition of an EEG signal. This process includes Haar, Symlets, Daubechies, Coiflets, Discrete Meyer, Biorthogonal and reverse Biorthogonal wavelet families in extracting five different brainwave sub-bands for a source localization. For this process, we used the Independent Compon…
