Steady-state visual evoked potentials (SSVEP) brain-computer interface (BCI) provides reliable responses leading to high accuracy and information throughput. But achieving high accuracy typically requires a relatively long time window of one second or more. Various methods were proposed to improve sub-second response accuracy through subject-specific training and calibration. Substantial performance improvements were achieved with tedious calibration and subject-specific training; resulting in the user's discomfort. So, we propose a training-free method by combining spatial-filtering and temporal alignment (CSTA) to recognize SSVEP responses in sub-second response time. CSTA exploits linear correlation and non-linear similarity between steady-state responses and stimulus templates with complementary fusion to achieve desirable performance improvements. We evaluated the performance of CSTA in terms of accuracy and Information Transfer Rate (ITR) in comparison with both training-based and training-free methods using two SSVEP data-sets. We observed that CSTA achieves the maximum mean accuracy of 97.43$\pm$2.26 % and 85.71$\pm$13.41 % with four-class and forty-class SSVEP data-sets respectively in sub-second response time in offline analysis. CSTA yields significantly higher mean performance (p<0.001) than the training-free method on both data-sets. Compared with training-based methods, CSTA shows 29.33$\pm$19.65 % higher mean accuracy with statistically significant differences in time window less than 0.5 s. In longer time windows, CSTA exhibits either better or comparable performance though not statistically significantly better than training-based methods. We show that the proposed method brings advantages of subject-independent SSVEP classification without requiring training while enabling high target recognition performance in sub-second response time.
In this work, a classification method for SSVEP-based BCI is proposed. The classification method uses features extracted by traditional SSVEP-based BCI methods and finds optimal discrimination thresholds for each feature to classify the targets. Optimising the thresholds is formalised as a maximisation task of a performance measure of BCIs called information transfer rate (ITR). However, instead of the standard method of calculating ITR, which makes certain assumptions about the data, a more general formula is derived to avoid incorrect ITR calculation when the standard assumptions are not met…
Swartz Center for Computational Neuroscience（斯沃茨计算神经科学中心）；Institute of Neural Computation（神经计算研究所）；University of California, San Diego（加州大学圣地亚哥分校）
Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have shown its robustness in facilitating high-efficiency communication. State-of-the-art training-based SSVEP decoding methods such as extended Canonical Correlation Analysis (CCA) and Task-Related Component Analysis (TRCA) are the major players that elevate the efficiency of the SSVEP-based BCIs through a calibration process. However, due to notable human variability across individuals and within individuals over time, calibration (training) data collection is non-negligible and often laborious and time-consu…
Objective: We used deep convolutional neural networks (DCNNs) to classify electroencephalography (EEG) signals in a steady-state visually evoked potentials (SSVEP) based single-channel brain-computer interface (BCI), which does not require calibration on the user. Methods: EEG signals were converted to spectrograms and served as input to train DCNNs using the transfer learning technique. We also modified and applied a data augmentation method, SpecAugment, generally employed for speech recognition. Furthermore, for comparison purposes, we classified the SSVEP dataset using Support-vector machi…
Boston University（波士顿大学）；Sabanci University（萨班哲大学）
Objective: Steady-state visually evoked potentials (SSVEPs), measured with EEG (electroencephalogram), yield decent information transfer rates (ITR) in brain-computer interface (BCI) spellers. However, the current high performing SSVEP BCI spellers in the literature require an initial lengthy and tiring user-specific training for each new user for system adaptation, including data collection with EEG experiments, algorithm training and calibration (all are before the actual use of the system). This impedes the widespread use of BCIs. To ensure practicality, we propose a highly novel target ide…
Southern University of Science and Technology（南方科技大学）
The key to electroencephalography (EEG)-based brain-computer interface (BCI) lies in neural decoding, and its accuracy can be improved by using hybrid BCI paradigms, that is, fusing multiple paradigms. However, hybrid BCIs usually require separate processing processes for EEG signals in each paradigm, which greatly reduces the efficiency of EEG feature extraction and the generalizability of the model. Here, we propose a two-stream convolutional neural network (TSCNN) based hybrid brain-computer interface. It combines steady-state visual evoked potential (SSVEP) and motor imagery (MI) paradigms…
U.S. Army Research Laboratory（美国陆军研究实验室）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Steady-State Visual Evoked Potentials (SSVEPs) are neural oscillations from the parietal and occipital regions of the brain that are evoked from flickering visual stimuli. SSVEPs are robust signals measurable in the electroencephalogram (EEG) and are commonly used in brain-computer interfaces (BCIs). However, methods for high-accuracy decoding of SSVEPs usually require hand-crafted approaches that leverage domain-specific knowledge of the stimulus signals, such as specific temporal frequencies in the visual stimuli and their relative spatial arrangement. When this knowledge is unavailable, suc…
iFuzzyTL: Interpretable Fuzzy Transfer Learning for SSVEP BCI System
University of Technology Sydney（悉尼科技大学）；Australian AI Institute（澳大利亚人工智能研究院）；School of Computer Science, Faculty of Engineering and Information Technology（工程与信息技术学院计算机科学学院）
Sabanci University（萨班哲大学）；Technical University of Munich（慕尼黑工业大学）
Objective: Target identification in brain-computer interface (BCI) spellers refers to the electroencephalogram (EEG) classification for predicting the target character that the subject intends to spell. When the visual stimulus of each character is tagged with a distinct frequency, the EEG records steady-state visually evoked potentials (SSVEP) whose spectrum is dominated by the harmonics of the target frequency. In this setting, we address the target identification and propose a novel deep neural network (DNN) architecture. Method: The proposed DNN processes the multi-channel SSVEP with convo…
Rethinking Self-Training Based Cross-Subject Domain Adaptation for SSVEP Classification
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（人工智能学院，北京邮电大学）；School of Computer Science and Technology, Beihang University（计算机科学与技术学院，北航）
Source-Free Domain Adaptation for SSVEP-based Brain-Computer Interfaces
Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA（电气与计算机工程系，波士顿大学）；Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany（人类认知与脑科学Max Planck研究所，莱比锡，德国）；Faculty of Engineering and Natural Sciences, Sabanci University, Istanbul, Turkey（工程与自然科学学院，萨班奇大学，伊斯坦布尔，土耳其）
School of Electrical and Computer Engineering（电气与计算机工程学院）
Objective: To propose novel SSVEP classification methodologies using deep neural networks (DNNs) and improve performances in single-channel and user-independent brain-computer interfaces (BCIs) with small data lengths. Approach: We propose the utilization of filter banks (creating sub-band components of the EEG signal) in conjunction with DNNs. In this context, we created three different models: a recurrent neural network (FBRNN) analyzing the time domain, a 2D convolutional neural network (FBCNN-2D) processing complex spectrum features and a 3D convolutional neural network (FBCNN-3D) analyzin…
Tshwane University of Technology（茨瓦内理工大学）；Université de Versailles Saint-Quentin（凡尔赛圣康丁大学）；Mensia Technologies（Mensia科技公司）
Riemannian geometry has been applied to Brain Computer Interface (BCI) for brain signals classification yielding promising results. Studying electroencephalographic (EEG) signals from their associated covariance matrices allows a mitigation of common sources of variability (electronic, electrical, biological) by constructing a representation which is invariant to these perturbations. While working in Euclidean space with covariance matrices is known to be error-prone, one might take advantage of algorithmic advances in information geometry and matrix manifold to implement methods for Symmetric…
U.S. Air Force Academy（美国空军学院）；OmniScience LLC（OmniScience有限责任公司）
Brain Computer Interface (BCI) technologies have the potential to improve the lives of millions of people around the world, whether through assistive technologies or clinical diagnostic tools. Despite advancements in the field, however, at present consumer and clinical viability remains low. A key reason for this is that many of the existing BCI deployments require substantial data collection per end-user, which can be cumbersome, tedious, and error-prone to collect. We address this challenge via a deep learning model, which, when trained across sufficient data from multiple subjects, offers r…
Single-trial classification of event-related potentials in electroencephalogram (EEG) signals is a very important paradigm of brain-computer interface (BCI). Because of individual differences, usually some subject-specific calibration data are required to tailor the classifier for each subject. Transfer learning has been extensively used to reduce such calibration data requirement, by making use of auxiliary data from similar/relevant subjects/tasks. However, all previous research assumes that all auxiliary data have been labeled. This paper considers a more general scenario, in which part of …
Many real-world brain-computer interface (BCI) applications rely on single-trial classification of event-related potentials (ERPs) in EEG signals. However, because different subjects have different neural responses to even the same stimulus, it is very difficult to build a generic ERP classifier whose parameters fit all subjects. The classifier needs to be calibrated for each individual subject, using some labeled subject-specific data. This paper proposes both online and offline weighted adaptation regularization (wAR) algorithms to reduce this calibration effort, i.e., to minimize the amount…
Huazhong University of Science and Technology（华中科技大学）；University of Technology Sydney（悉尼科技大学）；University of California San Diego（加州大学圣地亚哥分校）；Zürich University of Applied Sciences（苏黎世应用科技大学）
An electroencephalogram (EEG) based brain-computer interface (BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g., amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a computer. Most studies so far focused on making EEG-based BCI spellers faster and more reliable; however, few have considered their security. This study, for the first time, shows that P300 and steady-state visual evoked potential BCI spellers are very vulnerable, i.e., they can be severe…
Huazhong University of Science and Technology（华中科技大学）；Shanghai Jiao Tong University（上海交通大学）
A brain-computer interface (BCI) enables a user to communicate with a computer directly using brain signals. The most common non-invasive BCI modality, electroencephalogram (EEG), is sensitive to noise/artifact and suffers between-subject/within-subject non-stationarity. Therefore, it is difficult to build a generic pattern recognition model in an EEG-based BCI system that is optimal for different subjects, during different sessions, for different devices and tasks. Usually, a calibration session is needed to collect some training data for a new subject, which is time-consuming and user unfrie…
Université des Sciences et de la Technologie d’Oran Mohamed Boudiaf (USTO-MB)（奥兰穆罕默德·布迪亚夫科技大学（USTO-MB））
Speeding up the spelling in event-related potentials (ERP) based Brain-Computer Interfaces (BCI) requires eliciting strong brain responses in a short span of time, as much as the accurate classification of such evoked potentials remains challenging and imposes hard constraints for signal processing and machine learning techniques. Recent advances in stimulus presentation and deep learning showcased a promising direction in significantly improving the efficacy of those systems, in this study we propose the combination of colored inverted face stimulation with classification using convolutional …
Channel Reflection: Knowledge-Driven Data Augmentation for EEG-Based Brain-Computer Interfaces
Huazhong University of Science and Technology（华中科技大学）；Shenzhen Huazhong University of Science and Technology Research Institute（华中科技大学深圳研究院）；China Electronic System Technology Co., Ltd.（中国电子系统技术有限公司）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology（华中科技大学土木与水利工程学院）
U.S. Army Research Laboratory（美国陆军研究实验室）；DCS Corporation（DCS公司）；Columbia University（哥伦比亚大学）；Georgetown University（乔治城大学）
Brain computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional Neural Networks (CNNs), which have been used in computer vision and speech recognition, have successfully been applied to EEG-based BCIs; however, they have mainly be…
Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directions
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China（华中科技大学人工智能与自动化学院，武汉，中国）；Zhongguancun Academy, Beijing, China（中关村学院，北京，中国）
Online Adaptation via Dual-Stage Alignment and Self-Supervision for Fast-Calibration Brain-Computer Interfaces
School of Information, Shanxi University of Finance and Economics（山西财经大学信息学院）；Chinese Academy of Sciences（中国科学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation（多模态人工智能系统国家重点实验室，自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；Joint Laboratory of Intelligence Science and Technology, Institute of Systems Engineering, Macau University of Science and Technology（澳门科技大学智能科学与技术联合实验室）
The P300 event-related potential (ERP), evoked in scalp-recorded electroencephalography (EEG) by external stimuli, has proven to be a reliable response for controlling a BCI. The P300 component of an event related potential is thus widely used in brain-computer interfaces to translate the subjects' intent by mere thoughts into commands to control artificial devices. The main challenge in the classification of P300 trials in electroencephalographic (EEG) data is the low signal-to-noise ratio (SNR) of the P300 response. To overcome the low SNR of individual trials, it is common practice to avera…
In this article we present the results of our research related to the study of correlations between specific visual stimulation and the elicited brain's electro-physiological response collected by EEG sensors from a group of participants. We will look at how the various characteristics of visual stimulation affect the measured electro-physiological response of the brain and describe the optimal parameters found that elicit a steady-state visually evoked potential (SSVEP) in certain parts of the cerebral cortex where it can be reliably perceived by the electrode of the EEG device. After that, w…
Explainability of Recurrent Neural Networks for Enhancing P300-based Brain-Computer Interfaces
Grupo de Neurocomputación Biológica, Departamento de Ingeniería Informática, Escuela Politécnica Superior, Universidad Autónoma de Madrid（生物神经计算组，信息工程系，理工大学高级学院，马德里自治大学）；Grupo de Investigación en Electrónica y Telemática, Universidad Politécnica Salesiana（电子与电信研究组，萨利纳斯理工大学）
