In this study, a novel control algorithm for a P-300 based brain-computer interface is fully developed to control a 2-DoF robotic arm. Eight subjects including 5 men and 3 women, perform a 2-dimensional target tracking task in a simulated environment. Their EEG signals from visual cortex are recorded and P-300 components are extracted and evaluated to perform a real-time BCI based controller. The volunteer's intention is recognized and will be decoded as an appropriate command to control the cursor. The final goal of the system is to control a simulated robotic arm in a 2-dimensional space for writing some English letters. The results show that the system allows the robot end-effector to move between arbitrary positions in a point-to-point session with the desired accuracy. This model is tested on and compared with Dataset II of the BCI Competition. The best result is obtained with a multi-class SVM solution as the classifier, with a recognition rate of 97 percent, without pre-channel selection.
Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierarchically and transfer knowledge to student model. To this end, soft labels and distillation loss functions are applied to the student model training. According to experimental results, student model a…
The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further in…
The Brain-Inspired Cooperative Shared Control Framework for Brain-Machine Interface
Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua University（清华大学）
Westphalian University of Applied Sciences（威斯特伐利亚应用科学大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
Living a self-determined life independent of human caregivers or fully autonomous robots is a crucial factor for human dignity and the preservation of self-worth for people with motor impairments. Assistive robotic solutions - particularly robotic arms - are frequently deployed in domestic care, empowering people with motor impairments in performing ADLs independently. However, while assistive robotic arms can help them perform ADLs, currently available controls are highly complex and time-consuming due to the need to control multiple DoFs at once and necessary mode-switches. This work provide…
Brain-computer interface (BCI) decodes brain signals to understand user intention and status. Because of its simple and safe data acquisition process, electroencephalogram (EEG) is commonly used in non-invasive BCI. One of EEG paradigms, motor imagery (MI) is commonly used for recovery or rehabilitation of motor functions due to its signal origin. However, the EEG signals are an oscillatory and non-stationary signal that makes it difficult to collect and classify MI accurately. In this study, we proposed a band-power feature refining convolutional neural network (BFR-CNN) which is composed of …
We introduce Language-Informed Latent Actions (LILA), a framework for learning natural language interfaces in the context of human-robot collaboration. LILA falls under the shared autonomy paradigm: in addition to providing discrete language inputs, humans are given a low-dimensional controller $-$ e.g., a 2 degree-of-freedom (DoF) joystick that can move left/right and up/down $-$ for operating the robot. LILA learns to use language to modulate this controller, providing users with a language-informed control space: given an instruction like "place the cereal bowl on the tray," LILA may learn …
See-Control: A Multimodal Agent Framework for Smartphone Interaction with a Robotic Arm
University College London（伦敦大学学院）；Imperial College London（伦敦帝国学院）；Huawei Noah’s Ark Lab（华为诺亚实验室）；ShanghaiTech University（上海科技大学）；Southern University of Science（南方科技大学）
The fundamental frequency (F0) of human voice is generally controlled by changing the vocal fold parameters (including tension, length and mass), which in turn is manipulated by the muscle exciters, activated by the neural synergies. In order to begin investigating the neuromuscular F0 control pathway, we simulate a simple biomechanical arm prototype (instead of an artificial vocal tract) that tends to control F0 of an artificial sound synthesiser based on the elbow movements. The intended arm movements are decoded from the EEG signal inputs (collected simultaneously with the kinematic hand da…
