arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1910.02483cs.LGstat.ML
收藏

自动旋转感知器

Auto-Rotating Perceptrons

Pontificia Universidad Católica del Perú(秘鲁天主教大学)

Daniel Saromo, Elizabeth Villota, Edwin Villanueva

摘要:

本文提出了一种改进的感知器单元的设计,以减轻消失梯度的问题。当训练具有有界激活函数的深层多层感知器网络时,就会出现这种麻烦。新的神经元设计,命名为自动旋转感知器(ARP),有一个机制,以确保节点总是在激活函数的动态区域中操作,通过避免饱和的感知器。所提出的方法不改变在每个神经元处学习的推理结构。我们测试了在一些使用sigmoid激活函数的网络架构中使用ARP单元的效果。结果支持我们的假设,即具有ARP单元的神经网络可以获得比具有经典感知器的等效模型更好的学习性能。

英文摘要:

This paper proposes an improved design of the perceptron unit to mitigate the vanishing gradient problem. This nuisance appears when training deep multilayer perceptron networks with bounded activation functions. The new neuron design, named auto-rotating perceptron (ARP), has a mechanism to ensure that the node always operates in the dynamic region of the activation function, by avoiding saturation of the perceptron. The proposed method does not change the inference structure learned at each neuron. We test the effect of using ARP units in some network architectures which use the sigmoid activation function. The results support our hypothesis that neural networks with ARP units can achieve better learning performance than equivalent models with classic perceptrons.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑