arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.16763q-bio.NCcs.AIcs.NE
收藏

ELiSe:结构化循环网络中的高效序列学习

ELiSe: Efficient Learning of Sequences in Structured Recurrent Networks

University of Bern(伯尔尼大学);University of Zurich(苏黎世大学);ETH Zurich(苏黎世联邦理工学院);Catlab Engineering GmbH(Catlab工程有限公司)

Laura Kriener, Ben von Hünerbein, Kristin Spicher, Federico Benitez, Timo Gierlich, Arno Granier, Walter Senn, Mihai A. Petrovici

AI总结:

ELiSe利用皮层网络支架和树突区室,通过局部无相位突触可塑性高效学习复杂序列,在鸟类鸣唱模拟中展现灵活性和鲁棒性。

摘要:

行为可以被描述为由神经活动驱动的动作的时间序列。为了在神经网络中学习复杂的序列模式,过去活动的记忆需要在比单神经元活动弛豫时间长得多的时间尺度上持续存在。虽然循环网络能够产生这样的长瞬态,但训练这些网络是一个挑战。通过误差传播进行学习赋予诸如FORCE、RTRL或BPTT等模型显著的功能优势,但代价是生物合理性。虽然储层计算通过仅学习读出权重来规避这一问题,但它不能很好地随问题复杂性扩展。我们提出,皮层网络的两个显著结构特征可以缓解这些问题:学习开始时存在某种网络支架,以及存在树突区室以增强神经元信息存储和计算。我们由此产生的高效序列学习(ELiSe)模型基于这些特征,仅使用局部的、始终开启且无相位的突触可塑性来获取和重放复杂的非马尔可夫时空模式。我们在鸟类鸣唱学习的模拟中展示了ELiSe的能力,并证明了其相对于参数化的灵活性以及对外部干扰的鲁棒性。

英文摘要:

Behavior can be described as a temporal sequence of actions driven by neural activity. To learn complex sequential patterns in neural networks, memories of past activities need to persist on significantly longer timescales than the relaxation times of single-neuron activity. While recurrent networks can produce such long transients, training these networks is a challenge. Learning via error propagation confers models such as FORCE, RTRL or BPTT a significant functional advantage, but at the expense of biological plausibility. While reservoir computing circumvents this issue by learning only the readout weights, it does not scale well with problem complexity. We propose that two prominent structural features of cortical networks can alleviate these issues: the presence of a certain network scaffold at the onset of learning and the existence of dendritic compartments for enhancing neuronal information storage and computation. Our resulting model for Efficient Learning of Sequences (ELiSe) builds on these features to acquire and replay complex non-Markovian spatio-temporal patterns using only local, always-on and phase-free synaptic plasticity. We showcase the capabilities of ELiSe in a mock-up of birdsong learning, and demonstrate its flexibility with respect to parametrization, as well as its robustness to external disturbances.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑