arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1710.10403cs.LGstat.ML
收藏

可训练的反向传播函数转移矩阵

Trainable back-propagated functional transfer matrices

The University of Auckland(奥克兰大学);Beijing Forestry University(北京林业大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Griffith University(格里菲斯大学)

Cheng-Hao Cai, Yanyan Xu, Dengfeng Ke, Kaile Su, Jing Sun

摘要:

全连接神经网络节点之间的连接通常用权矩阵表示。在这篇文章中,引入函数转移矩阵作为权重矩阵的替代:函数转移矩阵使用具有可训练参数的实函数来表示节点之间的连接,而不是使用实权重。然后将多个功能转移矩阵与偏置向量和激活堆叠在一起以形成深度功能转移神经网络。这些神经网络可以在反向传播的框架内进行训练,基于delta规则和函数连接的错误传输规则的修正。在实验中,它被证明,修改后的规则可以用来训练的功能连接范围:20个不同的功能被应用到神经网络多达10个隐藏层,其中大多数获得高的测试精度的MNIST数据库。证明了具有记忆功能的函数转移矩阵可以粗略记忆400位的非循环序列。

英文摘要:

Connections between nodes of fully connected neural networks are usually represented by weight matrices. In this article, functional transfer matrices are introduced as alternatives to the weight matrices: Instead of using real weights, a functional transfer matrix uses real functions with trainable parameters to represent connections between nodes. Multiple functional transfer matrices are then stacked together with bias vectors and activations to form deep functional transfer neural networks. These neural networks can be trained within the framework of back-propagation, based on a revision of the delta rules and the error transmission rule for functional connections. In experiments, it is demonstrated that the revised rules can be used to train a range of functional connections: 20 different functions are applied to neural networks with up to 10 hidden layers, and most of them gain high test accuracies on the MNIST database. It is also demonstrated that a functional transfer matrix with a memory function can roughly memorise a non-cyclical sequence of 400 digits.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑