二元随机性赋能的高效神经形态深度学习实现超越软件精度
Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy
AI总结：
本文提出二元随机学习算法，通过对前向信号、激活函数导数、反向传播误差及权重更新进行二值化处理，在忆阻器硬件上实现超越软件基准的深度学习性能，并将乘累加运算能效提升三个数量级以上。
摘要：
深度学习需要对前向信号、反向传播误差和权重更新进行高精度处理。这是学习算法本身所固有的要求，因为梯度下降学习规则依赖于偏导数的链式乘积。然而，在使用带噪声的模拟忆阻器作为人工突触的硬件系统中实现深度学习具有挑战性，同时在生物学上也不具备合理性。基于忆阻器的实现通常会导致神经元电路成本过高，并对理想化突触器件提出严苛要求。在此，我们证明高精度要求并非必要，并且当这一要求被解除后，可以实现更高效的深度学习。我们提出了一种二元随机学习算法，该算法通过引入以下机制来修改所有基础神经网络运算：(i) 前向信号和激活函数导数的随机二值化，(ii) 反向传播误差的有符号二值化，以及 (iii) 步进式权重更新。通过软件仿真与硬件实验相结合的广泛混合方法，我们发现二元随机深度学习系统能够比使用高精度学习算法的软件基准提供更好的性能。此外，二元随机算法极大地简化了硬件中的神经网络运算，使乘累加运算的能效提升了三个数量级以上。
英文摘要：
Deep learning needs high-precision handling of forwarding signals, backpropagating errors, and updating weights. This is inherently required by the learning algorithm since the gradient descent learning rule relies on the chain product of partial derivatives. However, it is challenging to implement deep learning in hardware systems that use noisy analog memristors as artificial synapses, as well as not being biologically plausible. Memristor-based implementations generally result in an excessive cost of neuronal circuits and stringent demands for idealized synaptic devices. Here, we demonstrate that the requirement for high precision is not necessary and that more efficient deep learning can be achieved when this requirement is lifted. We propose a binary stochastic learning algorithm that modifies all elementary neural network operations, by introducing (i) stochastic binarization of both the forwarding signals and the activation function derivatives, (ii) signed binarization of the backpropagating errors, and (iii) step-wised weight updates. Through an extensive hybrid approach of software simulation and hardware experiments, we find that binary stochastic deep learning systems can provide better performance than the software-based benchmarks using the high-precision learning algorithm. Also, the binary stochastic algorithm strongly simplifies the neural network operations in hardware, resulting in an improvement of the energy efficiency for the multiply-and-accumulate operations by more than three orders of magnitudes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
