arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1612.05596cs.NEcs.AI
收藏

神经形态深度学习机器

Neuromorphic Deep Learning Machines

UC Irvine(加州大学尔湾分校);Intel Corporation(英特尔公司)

Emre Neftci, Charles Augustine, Somnath Paul, Georgios Detorakis

摘要:

类脑计算的一个持续的挑战是设计通用的和计算有效的推理和学习模型,这是兼容的空间和时间的限制,大脑。一种越来越流行和成功的方法是从深度神经网络中使用的推理和学习算法中获得灵感。然而,深度学习的主力,即梯度下降反向传播(BP)规则,通常依赖于存储在高精度存储器中的全网络信息的即时可用性,以及难以在神经形态硬件中实现的精确运算。值得注意的是,最近的研究表明,精确的反向传播权重对于学习深度表示并不重要。随机BP用随机权值代替反馈权值,并鼓励网络调整其前馈权值,以学习(随机)反馈权值的伪逆。在这些结果的基础上,我们展示了一种事件驱动的随机BP(eRBP)规则,该规则使用错误调制的突触可塑性来学习类脑计算硬件中的深度表示。该规则只需要一个加法和两个比较,每个突触的权重使用两室泄漏整合和消防(I&F)神经元,使其非常适合在数字或混合信号神经形态硬件的实现。我们的研究结果表明,使用eRBP,可以快速学习深度表示,与GPU上的人工神经网络模拟相比,可以实现几乎相同的分类精度,同时对学习期间的神经和突触状态量化具有鲁棒性。

英文摘要:

An ongoing challenge in neuromorphic computing is to devise general and computationally efficient models of inference and learning which are compatible with the spatial and temporal constraints of the brain. One increasingly popular and successful approach is to take inspiration from inference and learning algorithms used in deep neural networks. However, the workhorse of deep learning, the gradient descent Back Propagation (BP) rule, often relies on the immediate availability of network-wide information stored with high-precision memory, and precise operations that are difficult to realize in neuromorphic hardware. Remarkably, recent work showed that exact backpropagated weights are not essential for learning deep representations. Random BP replaces feedback weights with random ones and encourages the network to adjust its feed-forward weights to learn pseudo-inverses of the (random) feedback weights. Building on these results, we demonstrate an event-driven random BP (eRBP) rule that uses an error-modulated synaptic plasticity for learning deep representations in neuromorphic computing hardware. The rule requires only one addition and two comparisons for each synaptic weight using a two-compartment leaky Integrate & Fire (I&F) neuron, making it very suitable for implementation in digital or mixed-signal neuromorphic hardware. Our results show that using eRBP, deep representations are rapidly learned, achieving nearly identical classification accuracies compared to artificial neural network simulations on GPUs, while being robust to neural and synaptic state quantizations during learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑