arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.09440cs.NEcs.AI
收藏

突触动力学实现一阶自适应学习和权重对称

Synaptic Dynamics Realize First-order Adaptive Learning and Weight Symmetry

University of California at Santa Barbara(加州大学圣巴巴拉分校)

Yukun Yang, Peng Li

摘要:

基于自适应的一阶自适应优化方法,如Adam优化器,在人工网络的训练中很流行,取得了最先进的结果。这项工作试图回答的问题,它是否是可行的生物神经系统,采用这样的优化方法。为此,我们展示了一个实现的亚当优化器使用生物学上合理的机制突触。所提出的学习规则具有明确的生物学对应性,在时间上连续运行,并实现了与亚当相当的性能。此外,我们提出了一种新的方法,灵感来自神经科学中观察到的突触的易感性,以规避反向传播(BP)中的重量传输问题的生物难以置信。该方法只利用局部信息,不需要单独的训练阶段,建立并保持了前向和后向信号路径的权值对称性,适用于提出的生物学上合理的Adam学习规则。这些机制可能揭示生物突触动力学促进学习的方式。

英文摘要:

Gradient-based first-order adaptive optimization methods such as the Adam optimizer are prevalent in training artificial networks, achieving the state-of-the-art results. This work attempts to answer the question whether it is viable for biological neural systems to adopt such optimization methods. To this end, we demonstrate a realization of the Adam optimizer using biologically-plausible mechanisms in synapses. The proposed learning rule has clear biological correspondence, runs continuously in time, and achieves performance to comparable Adam's. In addition, we present a new approach, inspired by the predisposition property of synapses observed in neuroscience, to circumvent the biological implausibility of the weight transport problem in backpropagation (BP). With only local information and no separate training phases, this method establishes and maintains weight symmetry in the forward and backward signaling paths, and is applicable to the proposed biologically plausible Adam learning rule. These mechanisms may shed light on the way in which biological synaptic dynamics facilitate learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑