Motivated by the inconceivable capability of the human brain in simultaneously processing multi-modal signals and its real-time feedback to the outer world events, there has been a surge of interest in establishing a communication bridge between the human brain and a computer, which are referred to as Brain-computer Interfaces (BCI). To this aim, monitoring the electrical activity of brain through Electroencephalogram (EEG) has emerged as the prime choice for BCI systems. To discover the underlying and specific features of brain signals for different mental tasks, a considerable number of research works are developed based on statistical and data-driven techniques. However, a major bottleneck in the development of practical and commercial BCI systems is their limited performance when the number of mental tasks for classification is increased. In this work, we propose a new EEG processing and feature extraction paradigm based on Siamese neural networks, which can be conveniently merged and scaled up for multi-class problems. The idea of Siamese networks is to train a double-input neural network based on a contrastive loss-function, which provides the capability of verifying if two input EEG trials are from the same class or not. In this work, a Siamese architecture, which is developed based on Convolutional Neural Networks (CNN) and provides a binary output on the similarity of two inputs, is combined with OVR and OVO techniques to scale up for multi-class problems. The efficacy of this architecture is evaluated on a 4-class Motor Imagery (MI) dataset from BCI Competition IV-2a and the results suggest a promising performance compared to its counterparts.
Effective analysis of EEG signals for potential clinical applications remains a challenging task. So far, the analysis and conditioning of EEG have largely remained sex-neutral. This paper employs a machine learning approach to explore the evidence of sex effects on EEG signals, and confirms the generality of these effects by achieving successful sex prediction of resting-state EEG signals. We have found that the brain connectivity represented by the coherence between certain sensor channels are good predictors of sex.
BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals
Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室）；Department of Electronic Engineering, Tsinghua University, China（清华大学电子工程系）；Department of Psychology, University of Cambridge, UK（剑桥大学心理学系）；Speech Hearing and Phonetic Sciences, University College London, UK（伦敦大学学院语音听力与语音科学系）；MRC Cognition and Brain Sciences Unit, University of Cambridge, UK（剑桥大学MRC认知与脑科学单位）
ETH Zürich（苏黎世联邦理工学院）；University of Bologna（博洛尼亚大学）；Huawei Technologies（华为技术有限公司）
Motor-Imagery Brain--Machine Interfaces (MI-BMIs)promise direct and accessible communication between human brains and machines by analyzing brain activities recorded with Electroencephalography (EEG). Latency, reliability, and privacy constraints make it unsuitable to offload the computation to the cloud. Practical use cases demand a wearable, battery-operated device with low average power consumption for long-term use. Recently, sophisticated algorithms, in particular deep learning models, have emerged for classifying EEG signals. While reaching outstanding accuracy, these models often exceed…
National Tsing Hua University（台湾清华大学）；National Taiwan Ocean University（台湾海洋大学）；Elan Microelectronics Corporation（义隆电子股份有限公司）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Electroencephalography (EEG) signals are often contaminated with artifacts. It is imperative to develop a practical and reliable artifact removal method to prevent misinterpretations of neural signals and underperformance of brain-computer interfaces. This study developed a new artifact removal method, IC-U-Net, which is based on the U-Net architecture for removing pervasive EEG artifacts and reconstructing brain sources. The IC-U-Net was trained using mixtures of brain and non-brain sources decomposed by independent component analysis and employed an ensemble of loss functions to model comple…
Medical Center - University of Freiburg（弗莱堡大学医学中心）；University of Freiburg（弗莱堡大学）
In this manuscript, we investigate deep invertible networks for EEG-based brain signal decoding and find them to generate realistic EEG signals as well as classify novel signals above chance. Further ideas for their regularization towards better decoding accuracies are discussed.
BrainRVQ: A High-Fidelity EEG Foundation Model via Dual-Domain Residual Quantization and Hierarchical Autoregression
State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China（浙江大学工业控制技术状态重点实验室）；Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy（米兰理工学院电子、信息与生物工程系）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China（中国科学院深圳先进技术研究所）
Generative adversarial networks (GANs) are recently highly successful in generative applications involving images and start being applied to time series data. Here we describe EEG-GAN as a framework to generate electroencephalographic (EEG) brain signals. We introduce a modification to the improved training of Wasserstein GANs to stabilize training and investigate a range of architectural choices critical for time series generation (most notably up- and down-sampling). For evaluation we consider and compare different metrics such as Inception score, Frechet inception distance and sliced Wasser…
Representation and classification of Electroencephalography (EEG) brain signals are critical processes for their analysis in cognitive tasks. Particularly, extraction of discriminative features from raw EEG signals, without any pre-processing, is a challenging task. Motivated by nuclear norm, we observed that there is a significant difference between the variances of EEG signals captured from the same brain region when a subject performs different tasks. This observation lead us to use singular value decomposition for computing dominant variances of EEG signals captured from a certain brain re…
In recent years, multiple noninvasive imaging modalities have been used to develop a better understanding of the human brain functionality, including positron emission tomography, single-photon emission computed tomography, and functional magnetic resonance imaging, all of which provide brain images with millimeter spatial resolutions. Despite good spatial resolution, time resolution of these methods are poor and values are about seconds. Electroencephalography (EEG) is a popular non-invasive electrophysiological technique of relatively very high time resolution which is used to measure electr…
Electroencephalogram (EEG) has been one of the common neuromonitoring modalities for real-world brain-computer interfaces (BCIs) because of its non-invasiveness, low cost, and high temporal resolution. Recently, light-weight and portable EEG wearable devices based on low-density montages have increased the convenience and usability of BCI applications. However, loss of EEG decoding performance is often inevitable due to reduced number of electrodes and coverage of scalp regions of a low-density EEG montage. To address this issue, we introduce knowledge distillation (KD), a learning mechanism d…
Drivers cognitive and physiological states affect their ability to control their vehicles. Thus, these driver states are important to the safety of automobiles. The design of advanced driver assistance systems (ADAS) or autonomous vehicles will depend on their ability to interact effectively with the driver. A deeper understanding of the driver state is, therefore, paramount. EEG is proven to be one of the most effective methods for driver state monitoring and human error detection. This paper discusses EEG-based driver state detection systems and their corresponding analysis algorithms over t…
Despite knowing the reality of three-dimensional (3D) technology in the form of eye fatigue, this technology continues to be retained by people (especially the young community). To check what happens before and after watching a 2D and 3D movie and how this condition influences the human brain's power spectrum density (PSD), a five-member test group was arranged. In this study, electroencephalogram (EEG) was used as a neuroimaging method. EEG recordings of five individuals were taken both before and after watching 2D and 3D movies. After 2D/3D EEG recording, this record was divided into three s…
IMU-Enhanced EEG Motion Artifact Removal with Fine-Tuned Large Brain Models
Shu Chien-Gene Lay Department of Bioengineering and Institute for Neural Computation, University of California, San Diego (UCSD), La Jolla, CA 92093, USA（Shu Chien-Gene Lay 生物工程系和神经计算研究所，加州大学圣地亚哥分校（UCSD））
A trained T1 class Convolutional Neural Network (CNN) model will be used to examine its ability to successfully identify motor imagery when fed pre-processed electroencephalography (EEG) data. In theory, and if the model has been trained accurately, it should be able to identify a class and label it accordingly. The CNN model will then be restored and used to try and identify the same class of motor imagery data using much smaller sampled data in an attempt to simulate live data.
Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory Networks
School of Music, Jiangxi Normal University（江西师范大学音乐学院）；School of Electronic Information, HuZhou College（湖州学院电子信息学院）；San Francisco State University（旧金山州立大学）
Technische Hochschule Aschaffenburg（阿沙芬堡应用技术大学）；Universidade de São Paulo (USP)（圣保罗大学）
Mental disorders are among the leading causes of disability worldwide. The first step in treating these conditions is to obtain an accurate diagnosis, but the absence of established clinical tests makes this task challenging. Machine learning algorithms can provide a possible solution to this problem, as we describe in this work. We present a method for the automatic diagnosis of mental disorders based on the matrix of connections obtained from EEG time series and deep learning. We show that our approach can classify patients with Alzheimer's disease and schizophrenia with a high level of accu…
EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；East China University of Science and Technology（华东理工大学）；Wilmar International（丰益国际）
University of Rome Tor Vergata（罗马第二大学）；Drexel University（德雷塞尔大学）；University of Pisa（比萨大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
The brain computer interface (BCI) is a nonstimulatory direct and occasionally bidirectional communication link between the brain and a computer or an external device. Classically, EEG-based BCI algorithms have relied on models such as support vector machines and linear discriminant analysis or multiclass common spatial patterns. During the last decade, however, more sophisticated machine learning architectures, such as convolutional neural networks, recurrent neural networks, long short-term memory networks and gated recurrent unit networks, have been extensively used to enhance discriminabil…
EEG2MOTION: Towards Open-Vocabulary Human Motion Synthesis from Non-invasive Brain Signals
Zhejiang University（浙江大学）；Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The Hong Kong Polytechnic University（香港理工大学）；Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University（浙江大学教育部生物医学工程重点实验室）
Electroencephalography (EEG) analysis extracts critical information from brain signals, which has provided fundamental support for various applications, including brain-disease diagnosis and brain-computer interface. However, the real-time processing of large-scale EEG signals at high energy efficiency has placed great challenges for electronic processors on edge computing devices. Here, we propose the EEG opto-processor based on diffractive photonic computing units (DPUs) to effectively process the extracranial and intracranial EEG signals and perform epileptic seizure detection. The signals …
In this study, the Multivariate Empirical Mode Decomposition (MEMD) approach is applied to extract features from multi-channel EEG signals for mental state classification. MEMD is a data-adaptive analysis approach which is suitable particularly for multi-dimensional non-linear signals like EEG. Applying MEMD results in a set of oscillatory modes called intrinsic mode functions (IMFs). As the decomposition process is data-dependent, the IMFs vary in accordance with signal variation caused by functional brain activity. Among the extracted IMFs, it is found that those corresponding to high-oscill…
We present a novel solution to the problem of localization of MEG and EEG brain signals. The solution is sequential and iterative, and is based on minimizing the least-squares (LS)criterion by the Alternating Projection (AP) algorithm, which is well known in the context of array signal processing. Unlike existing scanning solutions belonging to the beamformer and multiple-signal classification (MUSIC) families, the algorithm has good performance in low signal-to-noise ratio (SNR) and can cope with closely spaced sources and any mixture of correlated sources. Results from simulated and experime…
CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Karlsruher Institut fur Technologie（卡尔斯鲁厄理工学院）
Classifying EEG responses to naturalistic acoustic stimuli is of theoretical and practical importance, but standard approaches are limited by processing individual channels separately on very short sound segments (a few seconds or less). Recent developments have shown classification for music stimuli (~2 mins) by extracting spectral components from EEG and using convolutional neural networks (CNNs). This paper proposes an efficient method to map raw EEG signals to individual songs listened for end-to-end classification. EEG channels are treated as a dimension of a [Channel x Sample] image tile…
Decoding brain signals has gained many attention and has found much applications in recent years such as Brain Computer Interfaces, communicating with controlling external devices using the user's intentions, occupies an emerging field with the potential of changing the world, with diverse applications from rehabilitation to human augmentation. This being said brain signal analysis, EEG brain signal analysis in particular, is a challenging task. With the advances and achievements in the field of deep learning in problem solving with using only raw data, few attempts has been carried in recent …
ETH Zürich（苏黎世联邦理工学院）；Nihon University（日本大学）；Huawei Technologies（华为技术有限公司）
In recent years, deep learning (DL) has contributed significantly to the improvement of motor-imagery brain-machine interfaces (MI-BMIs) based on electroencephalography(EEG). While achieving high classification accuracy, DL models have also grown in size, requiring a vast amount of memory and computational resources. This poses a major challenge to an embedded BMI solution that guarantees user privacy, reduced latency, and low power consumption by processing the data locally. In this paper, we propose EEG-TCNet, a novel temporal convolutional network (TCN) that achieves outstanding accuracy wh…
Unlike conventional data such as natural images, audio and speech, raw multi-channel Electroencephalogram (EEG) data are difficult to interpret. Modern deep neural networks have shown promising results in EEG studies, however finding robust invariant representations of EEG data across subjects remains a challenge, due to differences in brain folding structures. Thus, invariant representations of EEG data would be desirable to improve our understanding of the brain activity and to use them effectively during transfer learning. In this paper, we propose an approach to learn deep representations …
Many studies have explored brain signals during the performance of a memory task to predict later remembered items. However, prediction methods are still poorly used in real life and are not practical due to the use of electroencephalography (EEG) recorded from the scalp. Ear-EEG has been recently used to measure brain signals due to its flexibility when applying it to real world environments. In this study, we attempt to predict whether a shown stimulus is going to be remembered or forgotten using ear-EEG and compared its performance with scalp-EEG. Our results showed that there was no signif…
Huazhong University of Science and Technology（华中科技大学）
Deep learning has made significant breakthroughs in many fields, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, deep learning models are vulnerable to adversarial attacks, in which deliberately designed small perturbations are added to the benign input samples to fool the deep learning model and degrade its performance. This paper considers transferability-based black-box attacks, where the attacker trains a substitute model to approximate the target model, and then generates adversarial examples from the substitute model to attack the target model. Learn…
The aim of this paper is to design and construct an electroencephalograph (EEG) based brain-controlled wheelchair to provide a communication bridge from the nervous system to the external technical device for people of determination or individuals suffering from partial or complete paralysis. EEG is a technique that reads the activity of the brain by capturing brain signals non-invasively using a special EEG headset. The signals acquired go through pre-processing, feature extraction and classification. This technique allows human thoughts alone to be converted to control the wheelchair. The co…
Brain-Computer Interfaces (BCIs) enable converting the brain electrical activity of an interface user to the user commands. BCI research studies demonstrated encouraging results in different areas such as neurorehabilitation, control of artificial limbs, control of computer environments, communication and detection of diseases. Most of BCIs use scalp-electroencephalography (EEG), which is a non-invasive method to capture the brain activity. Although EEG monitoring devices are available in the market, these devices are generally lab-oriented and expensive. Day-to-day use of BCIs is impractical …
Recently, practical brain-computer interface is actively carried out, especially, in an ambulatory environment. However, the electroencephalography signals are distorted by movement artifacts and electromyography signals in ambulatory condition, which make hard to recognize human intention. In addition, as hardware issues are also challenging, ear-EEG has been developed for practical brain-computer interface and is widely used. However, ear-EEG still contains contaminated signals. In this paper, we proposed robust two-stream deep neural networks in walking conditions and analyzed the visual re…
The source localization of the human brain activities is an important resource for the recognition of cognitive state, medical disorders and a better understanding of the brain in general. In this study, we have compared 51 mother wavelets from 7 different wavelet families in a Stationary Wavelet transform (SWT) decomposition of an EEG signal. This process includes Haar, Symlets, Daubechies, Coiflets, Discrete Meyer, Biorthogonal and reverse Biorthogonal wavelet families in extracting five different brainwave sub-bands for a source localization. For this process, we used the Independent Compon…
Despite significant recent progress in the area of Brain-Computer Interface (BCI), there are numerous shortcomings associated with collecting Electroencephalography (EEG) signals in real-world environments. These include, but are not limited to, subject and session data variance, long and arduous calibration processes and predictive generalisation issues across different subjects or sessions. This implies that many downstream applications, including Steady State Visual Evoked Potential (SSVEP) based classification systems, can suffer from a shortage of reliable data. Generating meaningful and …
The diagnosis of Alzheimer's disease (AD) in routine clinical practice is most commonly based on subjective clinical interpretations. Quantitative electroencephalography (QEEG) measures have been shown to reflect neurodegenerative processes in AD and might qualify as affordable and thereby widely available markers to facilitate the objectivization of AD assessment. Here, we present a novel framework combining Riemannian tangent space mapping and elastic net regression for the development of brain atrophy markers. While most AD QEEG studies are based on small sample sizes and psychological test…
