arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.06381cs.NEcs.ET
收藏

基于电子突触器件的硬件深度神经网络自适应学习规则

Adaptive Learning Rule for Hardware-based Deep Neural Networks Using Electronic Synapse Devices

Suhwan Lim, Jong-Ho Bae, Jai-Ho Eum, Sungtae Lee, Chul-Heung Kim, Dongseok Kwon, Byung-Gook Park, Jong-Ho Lee

摘要:

在本文中,我们提出了一种基于反向传播(BP)算法的学习规则,该算法可应用于基于硬件的深度神经网络(HW-DNN),该网络使用具有离散和有限电导特性的电子设备。这种自适应学习规则支持硬件中的前向、后向传播以及权重更新,在实现高能效和高速深度神经网络的过程中很有帮助。在使用三层感知器网络的模拟中,我们根据电子突触设备的各种电导响应和权重更新方法来评估学习性能。它示出的学习精度是可比的,当使用基于软件的BP算法时,电子突触设备具有高动态范围的线性电导响应。此外,所提出的单向权值更新方法适用于具有非线性和有限电导响应的电子突触器件。由于这种权值更新方法可以弥补权值不对称更新的缺点,因此与其他方法相比,可以获得更好的精度。可应用于全硬件实现的该自适应学习规则还可补偿由于实际电子突触设备中的可能的设备到设备变化而导致的学习准确度的降低。

英文摘要:

In this paper, we propose a learning rule based on a back-propagation (BP) algorithm that can be applied to a hardware-based deep neural network (HW-DNN) using electronic devices that exhibit discrete and limited conductance characteristics. This adaptive learning rule, which enables forward, backward propagation, as well as weight updates in hardware, is helpful during the implementation of power-efficient and high-speed deep neural networks. In simulations using a three-layer perceptron network, we evaluate the learning performance according to various conductance responses of electronic synapse devices and weight-updating methods. It is shown that the learning accuracy is comparable to that obtained when using a software-based BP algorithm when the electronic synapse device has a linear conductance response with a high dynamic range. Furthermore, the proposed unidirectional weight-updating method is suitable for electronic synapse devices which have nonlinear and finite conductance responses. Because this weight-updating method can compensate the demerit of asymmetric weight updates, we can obtain better accuracy compared to other methods. This adaptive learning rule, which can be applied to full hardware implementation, can also compensate the degradation of learning accuracy due to the probable device-to-device variation in an actual electronic synapse device.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑