arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.06424cs.LGcs.AI
收藏

陀飞轮:一个物理上合理的神经架构

Tourbillon: a Physically Plausible Neural Architecture

University of California, Irvine(加州大学欧文分校);University of Hawai‘i at Mānoa(夏威夷大学马诺阿分校)

Mohammadamin Tavakoli, Peter Sadowski, Pierre Baldi

摘要:

在物理神经系统中,反向传播面临着许多障碍,包括:需要标记数据,违反局部学习原则,需要对称连接,以及缺乏模块化。陀飞轮是一个新的架构,解决了所有这些限制。在其核心,它由一堆圆形自动编码器组成,后面是一个输出层。循环自动编码器在自监督模式下通过再循环算法进行训练,顶层在监督模式下通过随机梯度下降进行训练,可以选择使用非对称连接将误差信息传播到整个堆栈。虽然陀飞轮架构主要是为了解决物理限制,而不是为了改善深度学习的当前工程应用,但我们在标准基准数据集上证明了其可行性,包括MNIST,Fashion MNIST和CIFAR10。我们表明,陀飞轮可以实现与反向传播训练的模型相当的性能,并优于使用其他物理上合理的算法(如反馈对齐)训练的模型。

英文摘要:

In a physical neural system, backpropagation is faced with a number of obstacles including: the need for labeled data, the violation of the locality learning principle, the need for symmetric connections, and the lack of modularity. Tourbillon is a new architecture that addresses all these limitations. At its core, it consists of a stack of circular autoencoders followed by an output layer. The circular autoencoders are trained in self-supervised mode by recirculation algorithms and the top layer in supervised mode by stochastic gradient descent, with the option of propagating error information through the entire stack using non-symmetric connections. While the Tourbillon architecture is meant primarily to address physical constraints, and not to improve current engineering applications of deep learning, we demonstrate its viability on standard benchmark datasets including MNIST, Fashion MNIST, and CIFAR10. We show that Tourbillon can achieve comparable performance to models trained with backpropagation and outperform models that are trained with other physically plausible algorithms, such as feedback alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑