arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1504.05767cs.NE
收藏

低分辨率突触权值神经网络的舍入方法

Rounding Methods for Neural Networks with Low Resolution Synaptic Weights

Lorenz K. Muller, Giacomo Indiveri

摘要:

在标准计算平台上模拟的神经网络算法通常利用具有浮点符号的高分辨率权重。然而,对于这种算法的专用硬件实现,具有低分辨率的定点突触权重是优选的。在这些算法中,通过标准舍入方法来降低权重的分辨率的基本方法会导致性能的急剧损失。为了进一步降低分辨率,在极端情况下甚至降低到二进制权重,需要更先进的技术。为此,我们提出了两种方法映射神经网络算法与高分辨率的权重,以相应的算法,工作与低分辨率的权重,并证明其性能大大优于标准舍入。我们进一步使用这些方法来研究三种常见的神经网络算法的性能固定的内存大小的权重矩阵与不同的权重分辨率。我们表明,专用的硬件系统,其技术要求非常低的重量分辨率(无论是电子或生物)原则上可以实现我们研究的算法。

英文摘要:

Neural network algorithms simulated on standard computing platforms typically make use of high resolution weights, with floating-point notation. However, for dedicated hardware implementations of such algorithms, fixed-point synaptic weights with low resolution are preferable. The basic approach of reducing the resolution of the weights in these algorithms by standard rounding methods incurs drastic losses in performance. To reduce the resolution further, in the extreme case even to binary weights, more advanced techniques are necessary. To this end, we propose two methods for mapping neural network algorithms with high resolution weights to corresponding algorithms that work with low resolution weights and demonstrate that their performance is substantially better than standard rounding. We further use these methods to investigate the performance of three common neural network algorithms under fixed memory size of the weight matrix with different weight resolutions. We show that dedicated hardware systems, whose technology dictates very low weight resolutions (be they electronic or biological) could in principle implement the algorithms we study.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑