超越黑箱:提升基于神经数据训练的Transformer的可解释性
Beyond Black Boxes: Enhancing Interpretability of Transformers Trained on Neural Data
AI总结:
本研究针对神经解码用Transformer决策透明度不足的问题,将稀疏自编码器(SAEs)引入小鼠视觉皮层钙成像视觉刺激预测Transformer框架,在保留原性能的同时获得了具备可解释特征选择性响应的隐藏单元,验证了内部表征的功能,实现了深度学习性能与可解释性的结合。
摘要:
Transformer模型已成为从神经活动中解码刺激与行为的当前最优模型,极大推动了神经科学研究的发展。但其决策过程若能具备更高透明度,将大幅提升其在科研与临床场景中的应用价值。稀疏自编码器(Sparse Autoencoders, SAEs)能生成对特定变量产生选择性响应的隐藏单元,从而提升可解释性,是颇具前景的解决方案。本研究将SAEs引入神经解码框架,对一个经训练后可从小鼠视觉皮层钙成像数据预测视觉刺激的Transformer进行增强。加入SAE后的Transformer模型保留了原有性能,同时其隐藏单元可对刺激朝向、遗传背景等可解释特征产生选择性响应。此外,消融与特定变量相关的单元会削弱模型处理该变量的能力,揭示了特定内部表征如何支撑下游计算。综上,这些结果表明将SAEs与Transformer相结合,可将现代深度学习的强大能力与科学理解和临床转化所必需的可解释性相结合。
英文摘要:
Transformer models have become state-of-the-art in decoding stimuli and behavior from neural activity, significantly advancing neuroscience research. Yet greater transparency in their decision-making processes would substantially enhance their utility in scientific and clinical contexts. Sparse autoencoders offer a promising solution by producing hidden units that respond selectively to specific variables, enhancing interpretability. Here, we introduce SAEs into a neural decoding framework by augmenting a transformer trained to predict visual stimuli from calcium imaging in the mouse visual cortex. The enhancement of the transformer model with an SAE preserved its original performance while yielding hidden units that selectively responded to interpretable features, such as stimulus orientation and genetic background. Furthermore, ablating units associated with a given variable impaired the model's ability to process that variable, revealing how specific internal representations support downstream computations. Together, these results demonstrate that integrating SAEs with transformers combines the power of modern deep learning with the interpretability essential for scientific understanding and clinical translation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。