arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2002.03428cs.LGstat.ML
收藏

利用双变量学习率改进神经网络学习

Improving Neural Network Learning Through Dual Variable Learning Rates

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Elizabeth Liner, Risto Miikkulainen

摘要:

本文介绍了一种新的神经网络训练方法:双变量学习率(DVLR)。基于行为心理学的见解,双学习率用于不同地强调正确和不正确的反应,从而使对网络的反馈更加具体。此外,学习率作为网络性能的函数而变化,从而使其更有效。DVLR在三种类型的网络上实现:前馈,卷积和残差,以及两个域:MNIST和CIFAR-10。结果表明,一致提高的准确性,表明DVLR是一个有前途的,心理动机的技术训练神经网络模型。

英文摘要:

This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR). Building on insights from behavioral psychology, the dual learning rates are used to emphasize correct and incorrect responses differently, thereby making the feedback to the network more specific. Further, the learning rates are varied as a function of the network's performance, thereby making it more efficient. DVLR was implemented on three types of networks: feedforward, convolutional, and residual, and two domains: MNIST and CIFAR-10. The results suggest a consistently improved accuracy, demonstrating that DVLR is a promising, psychologically motivated technique for training neural network models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑