Mental task identification and classification using single/limited channel(s) electroencephalogram (EEG) signals in real-time play an important role in the design of portable brain-computer interface (BCI) and neurofeedback (NFB) systems. However, the real-time recorded EEG signals are often contaminated with noises such as ocular artifacts (OAs) and muscle artifacts (MAs), which deteriorate the hand-crafted features extracted from EEG signal, resulting inadequate identification and classification of mental tasks. Therefore, we investigate the use of recent deep learning techniques which do not require any manual feature extraction or artifact suppression step. In this paper, we propose a light-weight one-dimensional convolutional neural network (1D-CNN) architecture for mental task identification and classification. The robustness of the proposed architecture is evaluated using artifact-free and artifact-contaminated EEG signals taken from two publicly available databases (i.e, Keirn and Aunon ($K$) database and EEGMAT ($E$) database) and in-house ($R$) database recorded using single-channel neurosky mindwave mobile 2 (MWM2) EEG headset in performing not only mental/non-mental binary task classification but also different mental/mental multi-tasks classification. Evaluation results demonstrate that the proposed architecture achieves the highest subject-independent classification accuracy of $99.7\%$ and $100\%$ for multi-class classification and pair-wise mental tasks classification respectively in database $K$. Further, the proposed architecture achieves subject-independent classification accuracy of $99\%$ and $98\%$ in database $E$ and the recorded database $R$ respectively. Comparative performance analysis demonstrates that the proposed architecture outperforms existing approaches not only in terms of classification accuracy but also in robustness against artifacts.
In this study, the Multivariate Empirical Mode Decomposition (MEMD) approach is applied to extract features from multi-channel EEG signals for mental state classification. MEMD is a data-adaptive analysis approach which is suitable particularly for multi-dimensional non-linear signals like EEG. Applying MEMD results in a set of oscillatory modes called intrinsic mode functions (IMFs). As the decomposition process is data-dependent, the IMFs vary in accordance with signal variation caused by functional brain activity. Among the extracted IMFs, it is found that those corresponding to high-oscill…
The recorded electroencephalography (EEG) signals are usually contaminated by many artifacts. In recent years, deep learning models have been used for denoising of electroencephalography (EEG) data and provided comparable performance with that of traditional techniques. However, the performance of the existing networks in electromyograph (EMG) artifact removal was limited and suffered from the over-fitting problem. Here we introduce a novel convolutional neural network (CNN) with gradually ascending feature dimensions and downsampling in time series for removing muscle artifacts in EEG data. C…
Brain computer interface (BCI) has been popular as a key approach to monitor our brains recent year. Mental states monitoring is one of the most important BCI applications and becomes increasingly accessible. However, the mental state prediction accuracy and generality through encephalogram (EEG) are not good enough for everyday use. Here in this paper we present a deep learning-based EEG decoding method to read mental states. We propose a novel 1D convolutional neural network with different filter lengths to capture different frequency bands information. To improve the prediction accuracy, we…
Deep Learning has impacted various fields especially in bio-medical applications. Deep learning algorithms work well with both structured and unstructured data. Especially, convolutional neural network work well with signal-based data like EEG data. These types of data may or may not follow a pattern in their data. Algorithms like CNN help feature engineering and simplistic interpretation of the data. These algorithms are also better in comparison to other algorithms when generalised to a data belonging to larger data set.
Brain-Computer Interfaces (BCI) based on motor imagery translate mental motor images recognized from the electroencephalogram (EEG) to control commands. EEG patterns of different imagination tasks, e.g. hand and foot movements, are effectively classified with machine learning techniques using band power features. Recently, also Convolutional Neural Networks (CNNs) that learn both effective features and classifiers simultaneously from raw EEG data have been applied. However, CNNs have two major drawbacks: (i) they have a very large number of parameters, which thus requires a very large number o…
University of Shanghai for Science and Technology（上海理工大学）；Oslo Metropolitan University（奥斯陆城市大学）
Mental fatigue increases the risk of operator error in language comprehension tasks. In order to prevent operator performance degradation, we used EEG signals to assess the mental fatigue of operators in human-computer systems. This study presents an experimental design for fatigue detection in language comprehension tasks. We obtained EEG signals from a 14-channel wireless EEG detector in 15 healthy participants. Each participant was given a cognitive test of a language comprehension task, in the form of multiple choice questions, in which pronoun references were selected between nominal and …
A Statistical Approach for Synthetic EEG Data Generation
College of Science and Engineering, James Cook University（科学与工程学院，詹姆斯库克大学）；College of Health Care Sciences, James Cook University（健康护理科学学院，詹姆斯库克大学）；College of Public Health, Medical, and Vet Sciences, James Cook University（公共卫生、医学与兽医学学院，詹姆斯库克大学）
Many research works indicate that EEG bands, specifically the alpha and theta bands, have been potentially helpful cognitive load indicators. However, minimal research exists to validate this claim. This study aims to assess and analyze the impact of the alpha-to-theta and the theta-to-alpha band ratios on supporting the creation of models capable of discriminating self-reported perceptions of mental workload. A dataset of raw EEG data was utilized in which 48 subjects performed a resting activity and an induced task demanding exercise in the form of a multitasking SIMKAP test. Band ratios wer…
In this article, we propose a sparse spectra graph convolutional network (SSGCNet) for solving Epileptic EEG signal classification problems. The aim is to achieve a lightweight deep learning model without losing model classification accuracy. We propose a weighted neighborhood field graph (WNFG) to represent EEG signals, which reduces the redundant edges between graph nodes. WNFG has lower time complexity and memory usage than the conventional solutions. Using the graph representation, the sequential graph convolutional network is based on a combination of sparse weight pruning technique and t…
Recent behavioral and electroencephalograph (EEG) studies have defined ways that auditory spatial attention can be allocated over large regions of space. As with most experimental studies, behavior EEG was averaged over 10s of minutes because identifying abstract feature spatial codes from raw EEG data is extremely challenging. The goal of this study is to design a deep learning model that can learn from raw EEG data and predict auditory spatial information on a trial-by-trial basis. We designed a convolutional neural networks (CNN) model to predict the attended location or other stimulus loca…
EuroMov Digital Health in Motion（EuroMov运动数字健康机构）；Univ. Montpellier（蒙彼利埃大学）；IMT Mines Ales（法国国立高等矿业电信学校联盟阿莱斯矿业学院）
In recent years, neuroscientists have been interested to the development of brain-computer interface (BCI) devices. Patients with motor disorders may benefit from BCIs as a means of communication and for the restoration of motor functions. Electroencephalography (EEG) is one of most used for evaluating the neuronal activity. In many computer vision applications, deep neural networks (DNN) show significant advantages. Towards to ultimate usage of DNN, we present here a shallow neural network that uses mainly two convolutional neural network (CNN) layers, with relatively few parameters and fast …
Fine-Tuning Large Language Models Using EEG Microstate Features for Mental Workload Assessment
School of Computer Science, Technological University Dublin, Dublin, Ireland（计算机科学学院，都柏林技术大学，都柏林，爱尔兰）；Artificial Intelligence and Cognitive Load Lab, Applied Intelligence Research Centre, School of Computer Science, Technological University Dublin, Dublin, Ireland（人工智能与认知负荷实验室，应用智能研究中心，计算机科学学院，都柏林技术大学，都柏林，爱尔兰）
It is well known that electroencephalograms (EEGs) often contain artifacts due to muscle activity, eye blinks, and various other causes. Detecting such artifacts is an essential first step toward a correct interpretation of EEGs. Although much effort has been devoted to semi-automated and automated artifact detection in EEG, the problem of artifact detection remains challenging. In this paper, we propose a convolutional neural network (CNN) enhanced by transformers using belief matching (BM) loss for automated detection of five types of artifacts: chewing, electrode pop, eye movement, muscle, …
This paper proposes a novel graph signal-based deep learning method for electroencephalography (EEG) and its application to EEG-based video identification. We present new methods to effectively represent EEG data as signals on graphs, and learn them using graph convolutional neural networks. Experimental results for video identification using EEG responses obtained while watching videos show the effectiveness of the proposed approach in comparison to existing methods. Effective schemes for graph signal representation of EEG are also discussed.
COMSATS University Islamabad（伊斯兰堡COMSATS大学）；University of Okara（奥卡拉大学）；Ca’ Foscari University of Venice（威尼斯大学）；Ege University（埃格大学）；Université Paris-Est Créteil (UPEC)（巴黎东克雷泰伊大学（UPEC））
Mental disorders represent critical public health challenges as they are leading contributors to the global burden of disease and intensely influence social and financial welfare of individuals. The present comprehensive review concentrate on the two mental disorders: Major depressive Disorder (MDD) and Bipolar Disorder (BD) with noteworthy publications during the last ten years. There is a big need nowadays for phenotypic characterization of psychiatric disorders with biomarkers. Electroencephalography (EEG) signals could offer a rich signature for MDD and BD and then they could improve under…
Decoding brain signals has gained many attention and has found much applications in recent years such as Brain Computer Interfaces, communicating with controlling external devices using the user's intentions, occupies an emerging field with the potential of changing the world, with diverse applications from rehabilitation to human augmentation. This being said brain signal analysis, EEG brain signal analysis in particular, is a challenging task. With the advances and achievements in the field of deep learning in problem solving with using only raw data, few attempts has been carried in recent …
The ability to perceive and recognize objects is fundamental for the interaction with the external environment. Studies that investigate them and their relationship with brain activity changes have been increasing due to the possible application in an intuitive brain-machine interface (BMI). In addition, the distinctive patterns when presenting different visual stimuli that make data differentiable enough to be classified have been studied. However, reported classification accuracy still low or employed techniques for obtaining brain signals are impractical to use in real environments. In this…
Non-invasive brain-computer interface technology has been developed for detecting human mental states with high performances. Detection of the pilots' mental states is particularly critical because their abnormal mental states could cause catastrophic accidents. In this study, we presented the feasibility of classifying distraction levels (namely, normal state, low distraction, and high distraction) by applying the deep learning method. To the best of our knowledge, this study is the first attempt to classify distraction levels under a flight environment. We proposed a model for classifying di…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper presents a novel graph convolutional neural network (GCNN)-based approach for improving the diagnosis of neurological diseases using scalp-electroencephalograms (EEGs). Although EEG is one of the main tests used for neurological-disease diagnosis, the sensitivity of EEG-based expert visual diagnosis remains at $\sim$50\%. This indicates a clear need for advanced methodology to reduce the false negative rate in detecting abnormal scalp-EEGs. In that context, we focus on the problem of distinguishing the abnormal scalp EEGs of patients with neurological diseases, which were originally…
Technische Hochschule Aschaffenburg（阿沙芬堡应用技术大学）；Universidade de São Paulo (USP)（圣保罗大学）
Mental disorders are among the leading causes of disability worldwide. The first step in treating these conditions is to obtain an accurate diagnosis, but the absence of established clinical tests makes this task challenging. Machine learning algorithms can provide a possible solution to this problem, as we describe in this work. We present a method for the automatic diagnosis of mental disorders based on the matrix of connections obtained from EEG time series and deep learning. We show that our approach can classify patients with Alzheimer's disease and schizophrenia with a high level of accu…
Recognizing the feelings of human beings plays a critical role in our daily communication. Neuroscience has demonstrated that different emotion states present different degrees of activation in different brain regions, EEG frequency bands and temporal stamps. In this paper, we propose a novel structure to explore the informative EEG features for emotion recognition. The proposed module, denoted by PST-Attention, consists of Positional, Spectral and Temporal Attention modules to explore more discriminative EEG features. Specifically, the Positional Attention module is to capture the activate re…
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
Critical task and cognition-based environments, such as in military and defense operations, aviation user-technology interaction evaluation on UI, understanding intuitiveness of a hardware model or software toolkit, etc. require an assessment of how much a particular task is generating mental workload on a user. This is necessary for understanding how those tasks, operations, and activities can be improvised and made better suited for the users so that they reduce the mental workload on the individual and the operators can use them with ease and less difficulty. However, a particular task can …
Brain-Computer Interface (BCI) system provides a pathway between humans and the outside world by analyzing brain signals which contain potential neural information. Electroencephalography (EEG) is one of most commonly used brain signals and EEG recognition is an important part of BCI system. Recently, convolutional neural networks (ConvNet) in deep learning are becoming the new cutting edge tools to tackle the problem of EEG recognition. However, training an effective deep learning model requires a big number of data, which limits the application of EEG datasets with a small number of samples.…
Tokyo University of Agriculture and Technology（东京农工大学）；Juntendo University Nerima Hospital（顺天堂大学练马医院）
Electroencephalography (EEG) is essential for the diagnosis of epilepsy, but it requires expertise and experience to identify abnormalities. It is thus crucial to develop automated models for the detection of abnormalities in EEGs related to epilepsy. This paper describes the development of a novel class of compact convolutional neural networks (CNNs) for detecting abnormal patterns and electrodes in EEGs for epilepsy. The designed model is inspired by a CNN developed for brain-computer interfacing called multichannel EEGNet (mEEGNet). Unlike the EEGNet, the proposed model, mEEGNet, has the sa…
Vidyasirimedhi Institute of Science & Engineering（维迪亚西里梅迪科学与工程学院）；Worcester Polytechnic Institute（伍斯特理工学院）；King Mongkut’s University of Technology Thonburi（国王科技大学吞武里分校）；Linköping University（林雪平大学）；Chulalongkorn University（朱拉隆功大学）
Electroencephalography (EEG) is another mode for performing Person Identification (PI). Due to the nature of the EEG signals, EEG-based PI is typically done while the person is performing some kind of mental task, such as motor control. However, few works have considered EEG-based PI while the person is in different mental states (affective EEG). The aim of this paper is to improve the performance of affective EEG-based PI using a deep learning approach. \textcolor{red}{We proposed a cascade of deep learning using a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Netwo…
INFANT Research Centre（INFANT研究中心）；University College Cork（科克大学）
Electroencephalography (EEG) is a valuable clinical tool for grading injury caused by lack of blood and oxygen to the brain during birth. This study presents a novel end-to-end architecture, using a deep convolutional neural network, that learns hierarchical representations within raw EEG data. The system classifies 4 grades of hypoxic-ischemic encephalopathy and is evaluated on a multi-channel EEG dataset of 63 hours from 54 newborns. The proposed method achieves a testing accuracy of 79.6% with one-step voting and 81.5% with two-step voting. These results show how a feature-free approach can…
Convolutional Neural Networks(CNNs) has achieved remarkable performance breakthrough in a variety of tasks. Recently, CNNs based methods that are fed with hand-extracted EEG features gradually produce a powerful performance on the EEG data based emotion recognition task. In this paper, we propose a novel convolutional layer allowing to adaptively extract effective data-driven spectrogram-like features from raw EEG signals, which we reference as scaling layer. Further, it leverages convolutional kernels scaled from one data-driven pattern to exposed a frequency-like dimension to address the sho…
EEG2GAIT: A Hierarchical Graph Convolutional Network for EEG-based Gait Decoding
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Institute of Neural Engineering, Graz University of Technology（技术大学神经工程研究所）；Center for Al in Medicine (C-AlM)（医学人工智能中心）
Mind wandering (MW) is a ubiquitous phenomenon which reflects a shift in attention from task-related to task-unrelated thoughts. There is a need for intelligent interfaces that can reorient attention when MW is detected due to its detrimental effects on performance and productivity. In this paper, we propose a deep learning model for MW detection using Electroencephalogram (EEG) signals. Specifically, we develop a channel-wise deep convolutional neural network (CNN) model to classify the features of focusing state and MW extracted from EEG signals. This is the first study that employs CNN to a…
In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this paper, we develop a novel convolutional neur…
Concordia Institute for Information Systems Engineering (CIISE)（康考迪亚信息系统工程研究所）；Concordia University（康考迪亚大学）
Motivated by the inconceivable capability of the human brain in simultaneously processing multi-modal signals and its real-time feedback to the outer world events, there has been a surge of interest in establishing a communication bridge between the human brain and a computer, which are referred to as Brain-computer Interfaces (BCI). To this aim, monitoring the electrical activity of brain through Electroencephalogram (EEG) has emerged as the prime choice for BCI systems. To discover the underlying and specific features of brain signals for different mental tasks, a considerable number of rese…
A new deep learning-based electroencephalography (EEG) signal analysis framework is proposed. While deep neural networks, specifically convolutional neural networks (CNNs), have gained remarkable attention recently, they still suffer from high dimensionality of the training data. Two-dimensional input images of CNNs are more vulnerable to be redundant versus one-dimensional input time-series of conventional neural networks. In this study, we propose a new dimensionality reduction framework for reducing the dimension of CNN inputs based on the tensor decomposition of the time-frequency represen…
Driver drowsiness is one of main factors leading to road fatalities and hazards in the transportation industry. Electroencephalography (EEG) has been considered as one of the best physiological signals to detect drivers drowsy states, since it directly measures neurophysiological activities in the brain. However, designing a calibration-free system for driver drowsiness detection with EEG is still a challenging task, as EEG suffers from serious mental and physical drifts across different subjects. In this paper, we propose a compact and interpretable Convolutional Neural Network (CNN) to disco…
NTT Communication Science Laboratories, NTT Corporation（NTT公司NTT通信科学实验室）；University of Oldenburg（奥尔登堡大学）
The performance of speech enhancement algorithms in a multi-speaker scenario depends on correctly identifying the target speaker to be enhanced. Auditory attention decoding (AAD) methods allow to identify the target speaker which the listener is attending to from single-trial EEG recordings. Aiming at enhancing the target speaker and suppressing interfering speakers, reverberation and ambient noise, in this paper we propose a cognitive-driven multi-microphone speech enhancement system, which combines a neural-network-based mask estimator, weighted minimum power distortionless response convolut…
In this paper, we propose a novel Convolutional Neural Network (CNN) approach for the classification of raw dry-EEG signals without any data pre-processing. To illustrate the effectiveness of our approach, we utilise the Steady State Visual Evoked Potential (SSVEP) paradigm as our use case. SSVEP can be utilised to allow people with severe physical disabilities such as Complete Locked-In Syndrome or Amyotrophic Lateral Sclerosis to be aided via BCI applications, as it requires only the subject to fixate upon the sensory stimuli of interest. Here we utilise SSVEP flicker frequencies between 10 …
ETH Zürich（苏黎世联邦理工学院）；Nihon University（日本大学）；Huawei Technologies（华为技术有限公司）
In recent years, deep learning (DL) has contributed significantly to the improvement of motor-imagery brain-machine interfaces (MI-BMIs) based on electroencephalography(EEG). While achieving high classification accuracy, DL models have also grown in size, requiring a vast amount of memory and computational resources. This poses a major challenge to an embedded BMI solution that guarantees user privacy, reduced latency, and low power consumption by processing the data locally. In this paper, we propose EEG-TCNet, a novel temporal convolutional network (TCN) that achieves outstanding accuracy wh…
Recently, visual perception (VP) and visual imagery (VI) paradigms are investigated in several brain-computer interface (BCI) studies. VP and VI are defined as a changing of brain signals when perceiving and memorizing visual information, respectively. These paradigms could be alternatives to the previous visual-based paradigms which have limitations such as fatigue and low information transfer rates (ITR). In this study, we analyzed VP and VI to investigate the possibility to control BCI. First, we conducted a time-frequency analysis with event-related spectral perturbation. In addition, two …
Brain-computer interface (BCI) decodes brain signals to understand user intention and status. Because of its simple and safe data acquisition process, electroencephalogram (EEG) is commonly used in non-invasive BCI. One of EEG paradigms, motor imagery (MI) is commonly used for recovery or rehabilitation of motor functions due to its signal origin. However, the EEG signals are an oscillatory and non-stationary signal that makes it difficult to collect and classify MI accurately. In this study, we proposed a band-power feature refining convolutional neural network (BFR-CNN) which is composed of …
