解码解码器：用于皮层语音解码的上下文序列到序列建模
Decoding the decoder: Contextual sequence-to-sequence modeling for intracortical speech decoding
AI总结：
本文提出基于Transformer的多任务序列到序列模型，用于从皮层记录中解码语音，通过引入Neural Hammer Scalpel模块提升鲁棒性和可解释性，实验显示其在语音解码任务中表现出色。
摘要：
语音脑机接口需要能够将皮层活动转化为语言输出的解码器，同时在数据有限和日常变化下保持鲁棒性。尽管先前高性能系统主要依赖于帧级音素解码结合下游语言模型，但尚不清楚上下文序列到序列解码对子词神经读取、鲁棒性和可解释性有何贡献。我们评估了一个多任务Transformer序列到序列模型，用于从区域6v皮层记录中尝试解码语音。该模型联合预测音素序列、词序列和辅助音频特征。为应对日常非平稳性，我们引入了Neural Hammer Scalpel（NHS）校准模块，结合全局对齐与特征级调制。我们进一步分析了留出日泛化和编码器和解码器的注意力模式。在Willett等人数据集上，所提模型实现了14.3%的音素错误率。词解码在直接解码时达到25.6%的WER，在候选生成和重评分时达到19.4%的WER。NHS相对于线性或无日特定变换显著提高了音素和词解码性能，而留出日实验显示随着时间距离增加，性能逐渐下降。注意力可视化揭示了编码器表示中的重复时间分块以及音素和词解码器对这些分块的不同使用。这些结果表明，上下文序列到序列建模可以提高从皮层语音信号到神经到音素读取的保真度，并表明基于注意力的分析可以生成有关神经语音证据如何分割和积累的有用假设。
英文摘要：
Speech brain--computer interfaces require decoders that translate intracortical activity into linguistic output while remaining robust to limited data and day-to-day variability. While prior high-performing systems have largely relied on framewise phoneme decoding combined with downstream language models, it remains unclear what contextual sequence-to-sequence decoding contributes to sublexical neural readout, robustness, and interpretability. We evaluated a multitask Transformer-based sequence-to-sequence model for attempted speech decoding from area 6v intracortical recordings. The model jointly predicts phoneme sequences, word sequences, and auxiliary acoustic features. To address day-to-day nonstationarity, we introduced the Neural Hammer Scalpel (NHS) calibration module, which combines global alignment with feature-wise modulation. We further analyzed held-out-day generalization and attention patterns in the encoder and decoders. On the Willett et al. dataset, the proposed model achieved a state-of-the-art phoneme error rate of 14.3%. Word decoding reached 25.6% WER with direct decoding and 19.4% WER with candidate generation and rescoring. NHS substantially improved both phoneme and word decoding relative to linear or no day-specific transform, while held-out-day experiments showed increasing degradation on unseen days with temporal distance. Attention visualizations revealed recurring temporal chunking in encoder representations and distinct use of these segments by phoneme and word decoders. These results indicate that contextual sequence-to-sequence modeling can improve the fidelity of neural-to-phoneme readout from intracortical speech signals and suggest that attention-based analyses can generate useful hypotheses about how neural speech evidence is segmented and accumulated over time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
