Recently, practical brain-computer interface is actively carried out, especially, in an ambulatory environment. However, the electroencephalography signals are distorted by movement artifacts and electromyography signals in ambulatory condition, which make hard to recognize human intention. In addition, as hardware issues are also challenging, ear-EEG has been developed for practical brain-computer interface and is widely used. However, ear-EEG still contains contaminated signals. In this paper, we proposed robust two-stream deep neural networks in walking conditions and analyzed the visual response EEG signals in the scalp and ear in terms of statistical analysis and brain-computer interface performance. We validated the signals with the visual response paradigm, steady-state visual evoked potential. The brain-computer interface performance deteriorated as 3~14% when walking fast at 1.6 m/s. When applying the proposed method, the accuracies increase 15% in cap-EEG and 7% in ear-EEG. The proposed method shows robust to the ambulatory condition in session dependent and session-to-session experiments.
Brain Computer Interface (BCI) can help patients of neuromuscular diseases restore parts of the movement and communication abilities that they have lost. Most of BCIs rely on mapping brain activities to device instructions, but limited number of brain activities decides the limited abilities of BCIs. To deal with the problem of limited ablility of BCI, this paper verified the feasibility of constructing BCI based on decoding imagined speech electroencephalography (EEG). As sentences decoded from EEG can have rich meanings, BCIs based on EEG decoding can achieve numerous control instructions. B…
Electroencephalogram (EEG) based brain-computer interface (BCI) systems are useful tools for clinical purposes like neural prostheses. In this study, we collected EEG signals related to grasp motions. Five healthy subjects participated in this experiment. They executed and imagined five sustained-grasp actions. We proposed a novel data augmentation method that increases the amount of training data using labels obtained from electromyogram (EMG) signals analysis. For implementation, we recorded EEG and EMG simultaneously. The data augmentation over the original EEG data concluded higher classif…
Brain-Computer Interface (BCI) system provides a pathway between humans and the outside world by analyzing brain signals which contain potential neural information. Electroencephalography (EEG) is one of most commonly used brain signals and EEG recognition is an important part of BCI system. Recently, convolutional neural networks (ConvNet) in deep learning are becoming the new cutting edge tools to tackle the problem of EEG recognition. However, training an effective deep learning model requires a big number of data, which limits the application of EEG datasets with a small number of samples.…
Transformers are groundbreaking architectures that have changed a flow of deep learning, and many high-performance models are developing based on transformer architectures. Transformers implemented only with attention with encoder-decoder structure following seq2seq without using RNN, but had better performance than RNN. Herein, we investigate the decoding technique for electroencephalography (EEG) composed of self-attention module from transformer architecture during imagined speech and overt speech. We performed classification of nine subjects using convolutional neural network based on EEGN…
Electroencephalogram (EEG) decoding aims to identify the perceptual, semantic, and cognitive content of neural processing based on non-invasively measured brain activity. Traditional EEG decoding methods have achieved moderate success when applied to data acquired in static, well-controlled lab environments. However, an open-world environment is a more realistic setting, where situations affecting EEG recordings can emerge unexpectedly, significantly weakening the robustness of existing methods. In recent years, deep learning (DL) has emerged as a potential solution for such problems due to it…
DS-MTNet:Structured Multi-Task EEG Decoding for Human-Machine Collaboration
School of Mechanical Science and Engineering, Huazhong University of Science and Technology（华中科技大学机械科学与工程学院）；School of Urban Transport and Logistics, Shenzhen Technology University（深圳职业技术大学城市交通与物流学院）
A deep neural network has been successfully applied to an electroencephalogram (EEG)-based brain-computer interface. However, in most studies, the correlation between EEG channels and inter-region relationships are not well utilized, resulting in sub-optimized spatial feature extraction. In this study, we propose an attention-based dual-stream 3D-convolutional neural network that can enhance spatial feature extraction by emphasizing the relationship between channels with dot product-based channel attention and 3D convolution. The proposed method showed superior performance than the comparative…
For utilization of robotic assistive devices in everyday life, means for detection and processing of erroneous robot actions are a focal aspect in the development of collaborative systems, especially when controlled via brain signals. Though, the variety of possible scenarios and the diversity of used robotic systems pose a challenge for error decoding from recordings of brain signals such as via EEG. For example, it is unclear whether humanoid appearances of robotic assistants have an influence on the performance. In this paper, we designed a study in which two different robots executed the s…
CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Karlsruher Institut fur Technologie（卡尔斯鲁厄理工学院）
Brain-computer interface (BCI) is a practical pathway to interpret users' intentions by decoding motor execution (ME) or motor imagery (MI) from electroencephalogram (EEG) signals. However, developing a BCI system driven by ME or MI is challenging, particularly in the case of containing continual and compound muscles movements. This study analyzes three grasping actions from EEG under both ME and MI paradigms. We also investigate the classification performance in offline and pseudo-online experiments. We propose a novel approach that uses muscle activity pattern (MAP) images for the convolutio…
Auditory Attention Decoding without Spatial Information: A Diotic EEG Study
Department of Electronic and Information Engineering, School of Engineering, The University of Osaka, Suita, Japan（大阪大学工学部电子信息工学科）；Graduate School of Engineering, The University of Osaka, Suita, Japan（大阪大学工学研究院）
Beihang University（北京航空航天大学）；The University of Sheffield（谢菲尔德大学）
Decoding EEG signals of different mental states is a challenging task for brain-computer interfaces (BCIs) due to nonstationarity of perceptual decision processes. This paper presents a novel boosted convolutional neural networks (ConvNets) decoding scheme for motor imagery (MI) EEG signals assisted by the multiwavelet-based time-frequency (TF) causality analysis. Specifically, multiwavelet basis functions are first combined with Geweke spectral measure to obtain high-resolution TF-conditional Granger causality (CGC) representations, where a regularized orthogonal forward regression (ROFR) alg…
AFPM: Alignment-based Frame Patch Modeling for Cross-Dataset EEG Decoding
Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（教育部图像处理与智能控制重点实验室，人工智能与自动化学院，华中科技大学）；Zhongguancun Academy（中关村学院）
Albert-Ludwigs-University Freiburg（弗莱堡阿尔伯特-路德维希大学）；University Medical Center Freiburg（弗莱堡大学医学中心）
The importance of robotic assistive devices grows in our work and everyday life. Cooperative scenarios involving both robots and humans require safe human-robot interaction. One important aspect here is the management of robot errors, including fast and accurate online robot-error detection and correction. Analysis of brain signals from a human interacting with a robot may help identifying robot errors, but accuracies of such analyses have still substantial space for improvement. In this paper we evaluate whether a novel framework based on deep convolutional neural networks (deep ConvNets) cou…
Visual Neural Decoding via Improved Visual-EEG Semantic Consistency
School of Computer Science and Technique, Tongji University（同济大学计算机科学与技术学院）；Shanghai Eye Disease Prevention and Treatment Center（上海市眼病防治中心）；Helen and John C. Hartmann Department of Electrical and Computer Engineering, New Jersey Institute of Technology（新泽西理工学院Helen and John C. Hartmann电气与计算机工程系）
Recently, practical brain-computer interface is actively carried out, especially, in an ambulatory environment. However, the electroencephalography (EEG) signals are distorted by movement artifacts and electromyography signals when users are moving, which make hard to recognize human intention. In addition, as hardware issues are also challenging, ear-EEG has been developed for practical brain-computer interface and has been widely used. In this paper, we proposed ensemble-based convolutional neural networks in ambulatory environment and analyzed the visual event-related potential responses in…
Decoding Workload and Agreement From EEG During Spoken Dialogue With Conversational AI
Chair of Neuroadaptive Human--Computer Interaction（神经适应性人机交互教授席位）；Brandenburg Technical University Cottbus--Senftenberg（勃兰登堡技术大学库滕堡-森芬根堡分校）；Auryal GmbH（Auryal公司）；Cottbus, Germany（库滕堡，德国）
EEG-Based Imagined Speech Decoding Using a Hybrid CNN-SNN Architecture
Doctoral School of Science, Technology, and Engineering, University of Granada（格拉纳达大学科学、技术与工程博士学院）；University of Technology Belfort-Montbéliard, SINERGIES（贝尔福-蒙贝利亚尔技术大学，协同实验室）；Department of Signal Theory, Telematics and Communications, University of Granada（格拉纳达大学信号理论、电信与通信系）；Department of Chemical Engineering, Stanford University（斯坦福大学化学工程系）；Department of Electrical and Computer Engineering, Aarhus University（奥尔胡斯大学电气与计算机工程系）
University Medical Center Freiburg（弗莱堡大学医学中心）；Alberts-Ludwigs-University Freiburg（阿尔伯特-路德维希-弗莱堡大学）
We recorded high-density EEG in a flanker task experiment (31 subjects) and an online BCI control paradigm (4 subjects). On these datasets, we evaluated the use of transfer learning for error decoding with deep convolutional neural networks (deep ConvNets). In comparison with a regularized linear discriminant analysis (rLDA) classifier, ConvNets were significantly better in both intra- and inter-subject decoding, achieving an average accuracy of 84.1 % within subject and 81.7 % on unknown subjects (flanker task). Neither method was, however, able to generalize reliably between paradigms. Visua…
Brain-computer interface (BCI) is the technology that enables the communication between humans and devices by reflecting status and intentions of humans. When conducting imagined speech, the users imagine the pronunciation as if actually speaking. In the case of decoding imagined speech-based EEG signals, complex task can be conducted more intuitively, but decoding performance is lower than that of other BCI paradigms. We modified our previous model for decoding imagined speech-based EEG signals. Ten subjects participated in the experiment. The average accuracy of our proposed method was 0.564…
Dual-TSST: A Dual-Branch Temporal-Spectral-Spatial Transformer Model for EEG Decoding
School of Software, Northwestern Polytechnical University（西北工业大学软件学院）；Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Yangtze River Delta Research Institute of Northwestern Polytechnical University（西北工业大学长三角研究院）
