一种改进运行时间的神经网络激活函数
A Modified Activation Function with Improved Run-Times For Neural Networks
摘要:
在本文中,我们提出了一个修改版本的双曲正切激活函数作为神经网络的学习单元生成器。该函数使用整数校准常数作为欧拉数e的近似值,基于二次实数公式(RNF)算法和对输入激活的自适应归一化约束,以避免消失梯度。我们使用假设和真实世界的数据集证明了所提出的修改的有效性,并表明通过使用此函数的学习算法可以实现更低的运行时间,从而提高训练过程中的速度和学习精度。
英文摘要:
In this paper we present a modified version of the Hyperbolic Tangent Activation Function as a learning unit generator for neural networks. The function uses an integer calibration constant as an approximation to the Euler number, e, based on a quadratic Real Number Formula (RNF) algorithm and an adaptive normalization constraint on the input activations to avoid the vanishing gradient. We demonstrate the effectiveness of the proposed modification using a hypothetical and real world dataset and show that lower run-times can be achieved by learning algorithms using this function leading to improved speed-ups and learning accuracies during training.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。