AI 中文总结
研究人员推出首个EEG-动作-文本数据集EEG2MOTION，提出EEG条件掩码动作模型EMMM，实现从无创脑信号生成多样连贯的全身体人体动作，为生成式开放词汇运动BCI开辟新方向。
AI 中文摘要
人体动作由分层运动系统调控，大脑提供高层意图，低层结构协调详细动力学。现有脑机接口（BCI）通常将此过程过度简化为受限分类或低维控制，无法捕捉自然动作的丰富性。由于稀疏神经信号与高维运动学之间存在显著跨模态差异，且缺乏大规模配对脑电（EEG）-动作数据集，实现开放词汇全身体动作合成仍具挑战性。为解决这一问题，我们推出EEG2MOTION，首个用于人体动作合成的EEG-动作-文本数据集，包含数千种动作的近2万对样本。利用该数据集，我们首先通过多模态对比学习证明，无创EEG嵌入可与文本、视频及动作表示有效对齐，以解码高层语义。随后，我们提出EEG条件掩码动作模型（EMMM），这是一种结合EEG编码器与动作解码器的生成框架，可直接从脑活动合成连续的全身体人体动作。实验结果显示，EMMM可从无创脑信号生成连贯且逼真的动作序列。据我们所知，这是首个从无创脑信号生成多样全身体人体动作的研究，为生成式开放词汇运动BCI开辟了新方向。项目页面见：this https URL。
英文摘要
Human motion is governed by a hierarchical motor system where the brain provides high-level intentions and lower-level structures coordinate detailed dynamics. Existing brain-computer interfaces (BCIs) typically oversimplify this into constrained classification or low-dimensional control, failing to capture the richness of natural movement. Bridging this gap to achieve open-vocabulary, full-body motion synthesis remains challenging due to the substantial cross-modal divergence between sparse neural signals and high-dimensional kinematics, as well as the lack of large-scale paired EEG-motion datasets. To address this, we introduce EEG2MOTION, the first EEG-motion-text dataset for human motion synthesis, comprising nearly 20,000 paired samples across thousands of motions. Using this dataset, we first demonstrate via multimodal contrastive learning that non-invasive EEG embeddings can be effectively aligned with text, video, and motion representations to decode high-level semantics. We then propose EEG-conditioned Masked Motion Model (EMMM), a generative framework that unites an EEG encoder with a motion decoder to synthesize continuous, full-body human motions directly from brain activity. Experimental results show that EMMM generates coherent and realistic motion sequences from non-invasive brain signals. To the best of our knowledge, this is the first work to generate diverse full-body human motions from non-invasive brain signals, opening a new direction toward generative and open-vocabulary motor BCIs. See our project page: https://yulom.github.io/EEG2MOTIONdemopage/.
