arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1503.05724stat.MLcs.LGcs.NE
收藏

一种用于加性和乘性相互作用之间平滑和可微过渡的神经传递函数

A Neural Transfer Function for a Smooth and Differentiable Transition Between Additive and Multiplicative Interactions

Technische Universität München(慕尼黑工业大学);fortiss(Fortiss研究所)

Sebastian Urban, Patrick van der Smagt

摘要:

结合加法和乘法神经单元的现有方法要么使用固定的操作分配,要么需要离散优化来确定神经元应该执行什么功能。这导致计算资源的低效分配或训练过程的计算复杂性的大幅增加。我们提出了一种新的,可参数化的传递函数的基础上的数学概念的非整数函数迭代,使每个神经元执行的操作是顺利的,最重要的是,微分之间的加法和乘法调整。这允许将加法和乘法之间的决策集成到标准反向传播训练过程中。

英文摘要:

Existing approaches to combine both additive and multiplicative neural units either use a fixed assignment of operations or require discrete optimization to determine what function a neuron should perform. This leads either to an inefficient distribution of computational resources or an extensive increase in the computational complexity of the training procedure. We present a novel, parameterizable transfer function based on the mathematical concept of non-integer functional iteration that allows the operation each neuron performs to be smoothly and, most importantly, differentiablely adjusted between addition and multiplication. This allows the decision between addition and multiplication to be integrated into the standard backpropagation training procedure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑