Practical brain-machine interfaces have been widely studied to accurately detect human intentions using brain signals in the real world. However, the electroencephalography (EEG) signals are distorted owing to the artifacts such as walking and head movement, so brain signals may be large in amplitude rather than desired EEG signals. Due to these artifacts, detecting accurately human intention in the mobile environment is challenging. In this paper, we proposed the reconstruction framework based on generative adversarial networks using the event-related potentials (ERP) during walking. We used a pre-trained convolutional encoder to represent latent variables and reconstructed ERP through the generative model which shape similar to the opposite of encoder. Finally, the ERP was classified using the discriminative model to demonstrate the validity of our proposed framework. As a result, the reconstructed signals had important components such as N200 and P300 similar to ERP during standing. The accuracy of reconstructed EEG was similar to raw noisy EEG signals during walking. The signal-to-noise ratio of reconstructed EEG was significantly increased as 1.3. The loss of the generative model was 0.6301, which is comparatively low, which means training generative model had high performance. The reconstructed ERP consequentially showed an improvement in classification performance during walking through the effects of noise reduction. The proposed framework could help recognize human intention based on the brain-machine interface even in the mobile environment.
Université des Sciences et de la Technologie d’Oran Mohamed Boudiaf (USTO-MB)（奥兰穆罕默德·布迪亚夫科技大学（USTO-MB））
Speeding up the spelling in event-related potentials (ERP) based Brain-Computer Interfaces (BCI) requires eliciting strong brain responses in a short span of time, as much as the accurate classification of such evoked potentials remains challenging and imposes hard constraints for signal processing and machine learning techniques. Recent advances in stimulus presentation and deep learning showcased a promising direction in significantly improving the efficacy of those systems, in this study we propose the combination of colored inverted face stimulation with classification using convolutional …
We present a mobile dataset obtained from electroencephalography (EEG) of the scalp and around the ear as well as from locomotion sensors by 24 participants moving at four different speeds while performing two brain-computer interface (BCI) tasks. The data were collected from 32-channel scalp-EEG, 14-channel ear-EEG, 4-channel electrooculography, and 9-channel inertial measurement units placed at the forehead, left ankle, and right ankle. The recording conditions were as follows: standing, slow walking, fast walking, and slight running at speeds of 0, 0.8, 1.6, and 2.0m/s, respectively. For ea…
Event-related potential (ERP) speller can be utilized in device control and communication for locked-in or severely injured patients. However, problems such as inter-subject performance instability and ERP-illiteracy are still unresolved. Therefore, it is necessary to predict classification performance before performing an ERP speller in order to use it efficiently. In this study, we investigated the correlations with ERP speller performance using a resting-state before an ERP speller. In specific, we used spectral power and functional connectivity according to four brain regions and five freq…
Reconstructing perceived images from human brain activity monitored by functional magnetic resonance imaging (fMRI) is hard, especially for natural images. Existing methods often result in blurry and unintelligible reconstructions with low fidelity. In this study, we present a novel approach for enhanced image reconstruction, in which existing methods for object decoding and image reconstruction are merged together. This is achieved by conditioning the reconstructed image to its decoded image category using a class-conditional generative adversarial network and neural style transfer. The resul…
PLA strategy support force information engineering university（中国人民解放军战略支援部队信息工程大学）
In the visual decoding domain, visually reconstructing presented images given the corresponding human brain activity monitored by functional magnetic resonance imaging (fMRI) is difficult, especially when reconstructing viewed natural images. Visual reconstruction is a conditional image generation on fMRI data and thus generative adversarial network (GAN) for natural image generation is recently introduced for this task. Although GAN-based methods have greatly improved, the fidelity and naturalness of reconstruction are still unsatisfactory due to the small number of fMRI data samples and the …
Intention decoding is an indispensable procedure in hands-free human-computer interaction (HCI). Conventional eye-tracking system using single-model fixation duration possibly issues commands ignoring users' real expectation. In the current study, an eye-brain hybrid brain-computer interface (BCI) interaction system was introduced for intention detection through fusion of multi-modal eye-track and ERP (a measurement derived from EEG) features. Eye-track and EEG data were recorded from 64 healthy participants as they performed a 40-min customized free search task of a fixed target icon among 25…
Zhejiang University（浙江大学）；Binjiang Institute of Zhejiang University（浙江大学滨江研究院）；Hangzhou Universal Medical Imaging Diagnostic Center（杭州全景医学影像诊断中心）
18F-fluorodeoxyglucose (18F-FDG) Positron Emission Tomography (PET) imaging usually needs a full-dose radioactive tracer to obtain satisfactory diagnostic results, which raises concerns about the potential health risks of radiation exposure, especially for pediatric patients. Reconstructing the low-dose PET (L-PET) images to the high-quality full-dose PET (F-PET) ones is an effective way that both reduces the radiation exposure and remains diagnostic accuracy. In this paper, we propose a resource-efficient deep learning framework for L-PET reconstruction and analysis, referred to as transGAN-S…
Toyota Motor Europe NV/SA（丰田汽车欧洲公司）；Camlin Italy S.r.l.（意大利卡姆林有限公司）；University of Parma（帕尔马大学）；National Research Council of Italy（意大利国家研究委员会）
Nowadays, the possibility to run advanced AI on embedded systems allows natural interaction between humans and machines, especially in the automotive field. We present a custom portable EEG-based Brain-Computer Interface (BCI) that exploits Event-Related Potentials (ERPs) induced with an oddball experimental paradigm to control the infotainment menu of a car. A preliminary evaluation of the system was performed on 10 participants in a standard laboratory setting and while driving on a closed private track. The task consisted of repeated presentations of 6 different menu icons in oddball fashio…
Due to the low signal-to-noise ratio and limited resolution of functional MRI data, and the high complexity of natural images, reconstructing a visual stimulus from human brain fMRI measurements is a challenging task. In this work, we propose a novel approach for this task, which we call Cortex2Image, to decode visual stimuli with high semantic fidelity and rich fine-grained detail. In particular, we train a surface-based convolutional network model that maps from brain response to semantic image features first (Cortex2Semantic). We then combine this model with a high-quality image generator (…
Dementia in the elderly has recently become the most usual cause of cognitive decline. The proliferation of dementia cases in aging societies creates a remarkable economic as well as medical problems in many communities worldwide. A recently published report by The World Health Organization (WHO) estimates that about 47 million people are suffering from dementia-related neurocognitive declines worldwide. The number of dementia cases is predicted by 2050 to triple, which requires the creation of an AI-based technology application to support interventions with early screening for subsequent ment…
This tutorial paper refers to the use of graph-theoretic concepts for analyzing brain signals. For didactic purposes it splits into two parts: theory and application. In the first part, we commence by introducing some basic elements from graph theory and stemming algorithmic tools, which can be employed for data-analytic purposes. Next, we describe how these concepts are adapted for handling evolving connectivity and gaining insights into network reorganization. Finally, the notion of signals residing on a given graph is introduced and elements from the emerging field of graph signal processin…
Vidyasirimedhi Institute of Science & Technology（维迪亚西里梅迪科学技术学院）；King Mongkut’s Institute of Technology Ladkrabang（国王科技大学（拉卡邦））；Kamnoetvidya Science Academy School（卡姆努特维迪亚科学学院）；Tokyo Institute of Technology（东京工业大学）
Brain biometrics based on electroencephalography (EEG) have been used increasingly for personal identification. Traditional machine learning techniques as well as modern day deep learning methods have been applied with promising results. In this paper we present EEG-BBNet, a hybrid network which integrates convolutional neural networks (CNN) with graph convolutional neural networks (GCNN). The benefit of the CNN in automatic feature extraction and the capability of GCNN in learning connectivity between EEG electrodes through graph representation are jointly exploited. We examine various connec…
Translating imagined speech from human brain activity into voice is a challenging and absorbing research issue that can provide new means of human communication via brain signals. Endeavors toward reconstructing speech from brain activity have shown their potential using invasive measures of spoken speech data, however, have faced challenges in reconstructing imagined speech. In this paper, we propose NeuroTalk, which converts non-invasive brain signals of imagined speech into the user's own voice. Our model was trained with spoken speech EEG which was generalized to adapt to the domain of ima…
Brain-Computer Interfaces (BCIs) are systems allowing people to interact with the environment bypassing the natural neuromuscular and hormonal outputs of the peripheral nervous system (PNS). These interfaces record a user's brain activity and translate it into control commands for external devices, thus providing the PNS with additional artificial outputs. In this framework, the BCIs based on the P300 Event-Related Potentials (ERP), which represent the electrical responses recorded from the brain after specific events or stimuli, have proven to be particularly successful and robust. The presen…
ART: Artifact Removal Transformer for Reconstructing Noise-Free Multichannel Electroencephalographic Signals
National Tsing Hua University（台湾清华大学）；ELAN Microelectronics Corporation（义隆电子股份有限公司）；National Yang Ming Chiao Tung University（阳明交通大学）；National Taipei University of Technology（台北科技大学）
University of Rochester（罗切斯特大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Recent advances in neuroimaging along with algorithmic innovations in statistical learning from network data offer a unique pathway to integrate brain structure and function, and thus facilitate revealing some of the brain's organizing principles at the system level. In this direction, we develop a supervised graph representation learning framework to model the relationship between brain structural connectivity (SC) and functional connectivity (FC) via a graph encoder-decoder system, where the SC is used as input to predict empirical FC. A trainable graph convolutional encoder captures direct …
National Tsing Hua University（台湾清华大学）；National Taiwan Ocean University（台湾海洋大学）；Elan Microelectronics Corporation（义隆电子股份有限公司）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Electroencephalography (EEG) signals are often contaminated with artifacts. It is imperative to develop a practical and reliable artifact removal method to prevent misinterpretations of neural signals and underperformance of brain-computer interfaces. This study developed a new artifact removal method, IC-U-Net, which is based on the U-Net architecture for removing pervasive EEG artifacts and reconstructing brain sources. The IC-U-Net was trained using mixtures of brain and non-brain sources decomposed by independent component analysis and employed an ensemble of loss functions to model comple…
EEG Right & Left Voluntary Hand Movement-based Virtual Brain-Computer Interfacing Keyboard Using Hybrid Deep Learning Approach
Nepal Engineering College（尼泊尔工程学院）；Pokhara University（博卡拉大学）；University of Delaware（特拉华大学）；Institute of Engineering Pulchowk Campus（工程学院普尔乔克校区）；Tribhuvan University（特里布文大学）；Nepal Research and Collaboration Center（尼泊尔研究与合作中心）
EEG-based recognition of activities and states involves the use of prior neuroscience knowledge to generate quantitative EEG features, which may limit BCI performance. Although neural network-based methods can effectively extract features, they often encounter issues such as poor generalization across datasets, high predicting volatility, and low model interpretability. Hence, we propose a novel lightweight multi-dimensional attention network, called LMDA-Net. By incorporating two novel attention modules designed specifically for EEG signals, the channel attention module and the depth attentio…
