脑-语言-动作（BLA）模型：用于机器人控制的语言条件化脑电图

中文摘要 AI 辅助

基于脑电图（EEG）的机器人控制通常被表述为直接分类问题，即将神经电信号映射到固定的离散动作集合。然而，EEG信号的可分性有限且噪声较高，使得该方法难以扩展到细粒度机器人控制空间。我们提出脑-语言-动作（BLA）模型，这是一种通过语言条件化神经表征解释以生成机器人动作的框架。在BLA中，一小组可可靠区分的脑状态可通过语言定义的控制映射动态关联到不同动作，从而使少量神经类别可应用于更大的全局动作空间。我们使用BCI Competition IV 2a数据集的运动想象EEG开发了用于无人机控制的概念验证BLA。该系统分两个阶段训练：首先，我们使用受试者特定的四分类运动想象分类评估多个候选EEG编码器架构，将250Hz、3.5秒、22通道的EEG样本转换为5个128维的脑标记嵌入；其次，将这些嵌入投影到预训练大语言模型（LLM）的嵌入空间，并与语言指令联合微调，以自回归生成结构化的三标记无人机动作。在4种神经状态与7种飞行动作组合之间的840种可能的语言定义映射中，所得BLA在评估期间达到90%的每标记准确率。这些结果初步证明，语言条件化可在无需相应增加直接可区分神经状态数量的情况下，扩展基于EEG的机器人接口的有效控制范围。

英文摘要

Electroencephalography (EEG)-based robotic control is commonly formulated as a direct classification problem, in which electrical neural signals are mapped to a fixed set of discrete actions. However, the limited separability and high noise of EEG signals make it difficult to scale this approach to fine-grained robotic control spaces. We introduce Brain-Language-Action (BLA) models, a framework in which language conditions the interpretation of neural representations for robotic action generation. In a BLA, a small set of reliably distinguishable brain states can be dynamically associated with different actions through a language-defined control mapping, allowing a small number of neural classes to apply to a larger global action space. We develop a proof-of-concept BLA for drone control using motor-imagery EEG from the BCI Competition IV 2a dataset. The system is trained in two stages. First, we evaluate multiple candidate EEG encoder architectures using subject-specific four-class motor-imagery classification, converting 250Hz, 3.5-second, 22-channel EEG samples into five 128-dimensional brain-token embeddings. Second, these embeddings are projected into the embedding space of a pretrained large language model (LLM) and jointly fine-tuned with language instructions to autoregressively generate structured three-token drone actions. Across 840 possible language-defined mappings between four neural states and seven flight action combinations, the resulting BLA achieves 90% per-token accuracy during evaluation. These results provide an initial demonstration that language conditioning can expand the effective control range of EEG-based robotic interfaces without requiring a corresponding increase in the number of directly distinguishable neural states.

发表机构

• VECTOR Labs（VECTOR实验室）

机构由 AI 辅助整理，请以论文原文为准。