Error Forward-Propagation：重用前馈连接来消除深度学习中的错误
Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learning
摘要：
我们引入了错误前向传播，这是一种生物学上合理的机制，可以通过网络向前传播错误反馈。对于大脑中的错误反馈，连接性的架构约束几乎被消除;系统的向后连接性不被使用或不需要来提供错误反馈。反馈作为一种将功劳分配给前向通路中早期神经元的手段，因为它们对最终输出的贡献被认为用于大脑的学习。大脑如何解决信用分配问题还不清楚。在机器学习中，误差反向传播是深度多层网络中信用分配的一种非常成功的机制。反向传播需要每个神经元的对称互易连接。从生物学的角度来看，没有证据表明存在这样的架构约束，这使得反向传播不可能用于大脑中的学习。通过使用随机反馈权重来减少这种架构约束。使用随机反馈权重的模型需要每个神经元的向后连接模式，但避免对称权重和相互连接。在本文中，我们实际上消除了这种结构上的限制，只需要一个有效的误差反馈的反向环路连接。我们建议重用的前向连接提供的错误反馈，通过馈送到输入接收层的输出。这种机制，错误前向传播，是一个合理的基础，错误反馈如何发生在大脑深处，独立于复杂的网络架构的功能，但支持。我们的实验表明，两个和三个隐藏层的递归神经网络可以使用误差前向传播MNIST和时尚MNIST数据集，实现$1.90\%$和$11\%$泛化误差分别训练。
英文摘要：
We introduce Error Forward-Propagation, a biologically plausible mechanism to propagate error feedback forward through the network. Architectural constraints on connectivity are virtually eliminated for error feedback in the brain; systematic backward connectivity is not used or needed to deliver error feedback. Feedback as a means of assigning credit to neurons earlier in the forward pathway for their contribution to the final output is thought to be used in learning in the brain. How the brain solves the credit assignment problem is unclear. In machine learning, error backpropagation is a highly successful mechanism for credit assignment in deep multilayered networks. Backpropagation requires symmetric reciprocal connectivity for every neuron. From a biological perspective, there is no evidence of such an architectural constraint, which makes backpropagation implausible for learning in the brain. This architectural constraint is reduced with the use of random feedback weights. Models using random feedback weights require backward connectivity patterns for every neuron, but avoid symmetric weights and reciprocal connections. In this paper, we practically remove this architectural constraint, requiring only a backward loop connection for effective error feedback. We propose reusing the forward connections to deliver the error feedback by feeding the outputs into the input receiving layer. This mechanism, Error Forward-Propagation, is a plausible basis for how error feedback occurs deep in the brain independent of and yet in support of the functionality underlying intricate network architectures. We show experimentally that recurrent neural networks with two and three hidden layers can be trained using Error Forward-Propagation on the MNIST and Fashion MNIST datasets, achieving $1.90\%$ and $11\%$ generalization errors respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
