均衡传播的连续权值更新和卷积结构
Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation
摘要：
平衡传播（EP）是一种生物启发的替代算法，用于训练神经网络。它适用于由静态输入x馈送的RNN，该输入x稳定到稳态，例如Hopfield网络。EP与BP的相似之处在于，在训练的第二阶段，误差信号在网络的层中向后传播，但与BP相反，EP的学习规则是空间局部的。然而，EP受到两个主要限制。一方面，由于其制定的实时动态，EP需要很长的仿真时间，这限制了其适用于实际任务。另一方面，EP的生物可扩展性受到其学习规则在时间上不是局部的事实的限制：突触更新在第二阶段的动态已经收敛之后执行，并且需要物理上不再可用的第一阶段的信息。我们的工作解决了这两个问题，并旨在扩大EP的频谱从标准的机器学习模型，以更逼真的生物神经网络。首先，我们提出了EP的离散时间公式，它可以简化方程，加快训练速度，并将EP扩展到CNN。我们的CNN模型实现了有史以来最好的性能报告MNIST与EP。使用相同的离散时间公式，我们引入了连续平衡传播（C-EP）：在训练的第二阶段，使用空间和时间的局部信息不断调整网络的权重。我们表明，在有限的缓慢变化的突触强度和小轻推，C-EP是等效的BPTT（定理1）。我们数值证明定理1和C-EP训练MNIST和推广到生物现实的情况下，神经网络与神经元之间的非对称连接。
英文摘要：
Equilibrium Propagation (EP) is a biologically inspired alternative algorithm to backpropagation (BP) for training neural networks. It applies to RNNs fed by a static input x that settle to a steady state, such as Hopfield networks. EP is similar to BP in that in the second phase of training, an error signal propagates backwards in the layers of the network, but contrary to BP, the learning rule of EP is spatially local. Nonetheless, EP suffers from two major limitations. On the one hand, due to its formulation in terms of real-time dynamics, EP entails long simulation times, which limits its applicability to practical tasks. On the other hand, the biological plausibility of EP is limited by the fact that its learning rule is not local in time: the synapse update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically. Our work addresses these two issues and aims at widening the spectrum of EP from standard machine learning models to more bio-realistic neural networks. First, we propose a discrete-time formulation of EP which enables to simplify equations, speed up training and extend EP to CNNs. Our CNN model achieves the best performance ever reported on MNIST with EP. Using the same discrete-time formulation, we introduce Continual Equilibrium Propagation (C-EP): the weights of the network are adjusted continually in the second phase of training using local information in space and time. We show that in the limit of slow changes of synaptic strengths and small nudging, C-EP is equivalent to BPTT (Theorem 1). We numerically demonstrate Theorem 1 and C-EP training on MNIST and generalize it to the bio-realistic situation of a neural network with asymmetric connections between neurons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
