arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.17641cs.LGeess.SP
收藏

RPNT:鲁棒预训练神经变换器——通用运动解码的路径

RPNT: Robust Pre-trained Neural Transformer -- A Pathway for Generalized Motor Decoding

University of Washington(华盛顿大学);Washington National Primate Research Center(华盛顿国家灵长类研究中心)

Hao Fang, Ryan A. Canfield, Tomohiro Ouchi, Beatrice Macagno, Eli Shlizerman, Amy L. Orsborn

AI总结:

本文提出RPNT模型,通过预训练实现鲁棒泛化,提升多任务、多主体和多会话的运动解码性能。

摘要:

脑运动解码旨在解读并翻译神经活动为行为。解码模型应能在不同变体上泛化,如不同脑区、实验 session、行为类型和主体的记录,这对于实际应用至关重要。当前解码模型只部分解决了这些挑战。在本工作中,我们开发了一个预训练神经变换器模型,RPNT - 鲁棒预训练神经变换器,通过预训练实现鲁棒泛化,从而在下游运动解码任务中实现有效的微调。我们通过系统研究哪些变换块适合神经尖峰活动建模来实现所提出的 RPNT 架构,因为其他模态(如文本和图像)模型的组件无法直接转移到神经数据。最终的 RPNT 架构包含三个独特的组件:1) 多维旋转位置嵌入,用于聚合实验元数据,如脑区坐标、session ID 和行为类型;2) 基于上下文的注意力机制,通过在全局注意力上使用卷积核来学习局部时间结构,以处理神经群体活动的非平稳性;3) 鲁棒的自监督学习目标,结合随机因果掩码策略和对比表示。我们预训练了两个版本的 RPNT 在具有显著泛化挑战的不同数据集上:a) 多 session、多任务、多主体微电极基准;b) 使用高密度 Neuropixel 1.0 探针进行多 site 记录,从许多皮层位置。预训练后,我们评估了 RPNT 在跨 session、跨类型、跨主体和跨 site 的下游行为解码任务中的泛化能力。我们的 RPNT 在这些任务中始终优于现有的解码模型。

英文摘要:

Brain motor decoding aims to interpret and translate neural activity into behaviors. Decoding models should generalize across variations, such as recordings from different brain sites, experimental sessions, behavior types, and subjects, will be critical for real-world applications. Current decoding models only partially address these challenges. In this work, we develop a pretrained neural transformer model, RPNT - Robust Pretrained Neural Transformer, designed to achieve robust generalization through pretraining, which in turn enables effective finetuning for downstream motor decoding tasks. We achieved the proposed RPNT architecture by systematically investigating which transformer building blocks could be suitable for neural spike activity modeling, since components from models developed for other modalities, such as text and images, do not transfer directly to neural data. The final RPNT architecture incorporates three unique enabling components: 1) Multidimensional rotary positional embedding to aggregate experimental metadata such as site coordinates, session ids and behavior types; 2) Context-based attention mechanism via convolution kernels operating on global attention to learn local temporal structures for handling non-stationarity of neural population activity; 3) Robust self-supervised learning objective with stochastic causal masking strategies and contrastive representations. We pretrained two versions of RPNT on distinct datasets that present significant generalization challenges: a) Multi-session, multi-task, and multi-subject microelectrode benchmark; b) Multi-site recordings using high-density Neuropixel 1.0 probes from many cortical locations. After pretraining, we evaluated RPNT generalization on cross-session, cross-type, cross-subject, and cross-site downstream behavior decoding tasks. Our RPNT consistently outperforms the existing decoding models on these tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑