arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.02549cs.NE
收藏

脉冲神经网络的小批量处理

Minibatch Processing in Spiking Neural Networks

Daniel J. Saunders, Cooper Sigrist, Kenneth Chaney, Robert Kozma, Hava T. Siegelmann

摘要:

脉冲神经网络(SNN)是一种很有前途的候选生物启发和能量有效的计算。然而,它们的模拟是出了名的耗时,并且可能被视为开发具有潜在部署在神经形态硬件平台上的竞争性训练方法的瓶颈。为了解决这个问题,我们提供了一个小批处理的实现应用于基于时钟的SNN模拟,导致大幅增加的数据吞吐量。据我们所知,这是脉冲神经网络模拟器中小批处理的第一个通用实现,该模拟器与任意神经元和突触模型一起工作。我们展示了几乎恒定的时间缩放与批量大小的模拟设置(高达GPU内存限制),并展示了大批量大小在两个SNN应用程序域的有效性,导致$\approx$ 880倍和$\approx$ 24倍的挂钟时间分别减少。不同的参数减少技术,以产生不同的学习结果,在模拟的网络训练与尖峰时间依赖的可塑性。机器学习从业者和生物建模者都可以从这种方法所能实现的大幅减少的模拟时间和增加的迭代速度中受益。可以在https://github.com/djsaunde/snn-minibatch上找到本文中再现基准测试和实验结果的代码。

英文摘要:

Spiking neural networks (SNNs) are a promising candidate for biologically-inspired and energy efficient computation. However, their simulation is notoriously time consuming, and may be seen as a bottleneck in developing competitive training methods with potential deployment on neuromorphic hardware platforms. To address this issue, we provide an implementation of mini-batch processing applied to clock-based SNN simulation, leading to drastically increased data throughput. To our knowledge, this is the first general-purpose implementation of mini-batch processing in a spiking neural networks simulator, which works with arbitrary neuron and synapse models. We demonstrate nearly constant-time scaling with batch size on a simulation setup (up to GPU memory limits), and showcase the effectiveness of large batch sizes in two SNN application domains, resulting in $\approx$880X and $\approx$24X reductions in wall-clock time respectively. Different parameter reduction techniques are shown to produce different learning outcomes in a simulation of networks trained with spike-timing-dependent plasticity. Machine learning practitioners and biological modelers alike may benefit from the drastically reduced simulation time and increased iteration speed this method enables. Code to reproduce the benchmarks and experimental findings in this paper can be found at https://github.com/djsaunde/snn-minibatch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑