受生物启发的Mamba:选择性状态空间模型中的时间局部性与生物可信学习
Bio-Inspired Mamba: Temporal Locality and Bioplausible Learning in Selective State Space Models
AI总结:
本研究提出结合生物学习原理与Mamba架构的在线学习框架BIM,通过融合RTRL与类STDP局部规则解决脉冲神经网络训练难题,在多任务上取得媲美传统方法的性能,提升能效并为神经形态实现与生物神经机制研究提供参考。
摘要:
本文提出了受生物启发的Mamba(Bio-Inspired Mamba,简称BIM),这是一种面向选择性状态空间模型的新型在线学习框架,将生物学习原理与Mamba架构相结合。BIM将实时循环学习(Real-Time Recurrent Learning,RTRL)与类似脉冲时序依赖可塑性(Spike-Timing-Dependent Plasticity,STDP)的局部学习规则相结合,解决了脉冲神经网络训练中的时间局部性与生物可信性难题。该方法利用了时间反向传播与STDP之间的内在联系,提供了一种计算高效的替代方案,同时保留了捕获长距离依赖的能力。我们在语言建模、语音识别和生物医学信号分析任务上对BIM进行了评估,结果表明其在遵循生物学习原理的同时,性能可与传统方法相媲美。实验结果显示该方法提升了能效,且具备神经形态硬件实现的潜力。BIM不仅推动了生物可信机器学习领域的发展,还为探究生物神经网络中的时间信息处理机制提供了思路。
英文摘要:
This paper introduces Bio-Inspired Mamba (BIM), a novel online learning framework for selective state space models that integrates biological learning principles with the Mamba architecture. BIM combines Real-Time Recurrent Learning (RTRL) with Spike-Timing-Dependent Plasticity (STDP)-like local learning rules, addressing the challenges of temporal locality and biological plausibility in training spiking neural networks. Our approach leverages the inherent connection between backpropagation through time and STDP, offering a computationally efficient alternative that maintains the ability to capture long-range dependencies. We evaluate BIM on language modeling, speech recognition, and biomedical signal analysis tasks, demonstrating competitive performance against traditional methods while adhering to biological learning principles. Results show improved energy efficiency and potential for neuromorphic hardware implementation. BIM not only advances the field of biologically plausible machine learning but also provides insights into the mechanisms of temporal information processing in biological neural networks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。