基于语义感知的对比学习在脑电文本生成与课程学习中的应用
Semantic-aware Contrastive Learning for Electroencephalography-to-Text Generation with Curriculum Learning
摘要：
由于脑机接口（BCI）的巨大潜力，直接从脑电信号中生成自然文本的脑电文本生成（EEG-to-Text generation，EEG-to-Text）技术近年来受到越来越多的关注。然而，受试者相关的EEG表示和语义相关的文本表示之间的显着差异对这项任务提出了很大的挑战。为了缓解这一挑战，我们设计了一个课程语义感知的对比学习策略（C-SCL），有效地重新校准的主题依赖的EEG表示的语义依赖的EEG表示，从而减少了差异。具体来说，我们的C-SCL将语义相似的EEG表示拉在一起，同时将不同的EEG表示分开。此外，为了引入更多有意义的对比对，我们谨慎地使用课程学习，不仅要制作有意义的对比对，而且要使学习过程循序渐进。我们在ZuCo基准测试上进行了大量的实验，我们的方法结合了不同的模型和架构，在三种类型的指标上显示出稳定的改进，同时实现了新的最先进的水平。进一步的研究证明了它不仅在单主题和低资源设置中的优越性，而且在zero-shot设置中具有强大的泛化能力。
英文摘要：
Electroencephalography-to-Text generation (EEG-to-Text), which aims to directly generate natural text from EEG signals has drawn increasing attention in recent years due to the enormous potential for Brain-computer interfaces (BCIs). However, the remarkable discrepancy between the subject-dependent EEG representation and the semantic-dependent text representation poses a great challenge to this task. To mitigate this challenge, we devise a Curriculum Semantic-aware Contrastive Learning strategy (C-SCL), which effectively re-calibrates the subject-dependent EEG representation to the semantic-dependent EEG representation, thus reducing the discrepancy. Specifically, our C-SCL pulls semantically similar EEG representations together while pushing apart dissimilar ones. Besides, in order to introduce more meaningful contrastive pairs, we carefully employ curriculum learning to not only craft meaningful contrastive pairs but also make the learning progressively. We conduct extensive experiments on the ZuCo benchmark and our method combined with diverse models and architectures shows stable improvements across three types of metrics while achieving the new state-of-the-art. Further investigation proves not only its superiority in both the single-subject and low-resource settings but also its robust generalizability in the zero-shot setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
