ADSEQ:一种用于脉冲神经网络中脉冲事件传递的延迟感知自动求导兼容框架
ADSEQ: A delay-aware autograd-compatible framework for spike-event delivery in SNNs
AI总结:
研究针对脉冲神经网络模拟和训练问题,提出通过推导含延迟的脉冲事件队列梯度计算并实现为ADSEQ框架,经多平台基准测试,发现队列设计影响性能,还给出性能 - 精度权衡方法及未来改进方向。
摘要:
脉冲神经网络(SNN)对于计算神经科学和神经形态机器学习至关重要,需要高效模拟和基于梯度的训练。虽然人工智能加速器能显著加速,但基于梯度的SNN通常用密集、内存占用大的数据结构实现稀疏脉冲事件。现有精确梯度方法缺乏通用性,当前模拟器常忽略或低效处理延迟脉冲。我们通过推导包含延迟的脉冲事件队列的梯度计算,并将其实现为内存高效、可自动求导的脉冲事件队列(ADSEQ)来解决此问题。在CPU、GPU、TPU和LPU平台上对其进行基准测试。我们发现队列设计对性能影响很大。CPU在传统基于树或FIFO实现中表现良好,GPU在较小模拟中使用环形缓冲区表现出色,但在更高内存压力下更喜欢更稀疏的数据结构。TPU似乎更喜欢基于排序内在函数的实现。选择性脉冲丢弃提供了简单的性能 - 精度权衡,未来自动求导框架适应不同的原始/切线数据结构可能会增强这种权衡。
英文摘要:
Spiking neural networks (SNNs), central to computational neuroscience and neuromorphic machine learning (ML), require efficient simulation and gradient-based training. While AI accelerators offer promising speedups, gradient-based SNNs typically implement sparse spike events using dense, memory-heavy data-structures. Existing exact gradient methods lack generality, and current simulators often omit or inefficiently handle delayed spikes. We address this by deriving gradient computation through spike event queues, including delays, and implementing this into memory-efficient, autodifferentiable spike event queues (ADSEQ). These are benchmarked across CPU, GPU, TPU, and LPU platforms. We find that queue design strongly shapes performance. CPUs, as expected, perform well with traditional tree-based or FIFO implementations, while GPUs excel with ring buffers for smaller simulations, yet under higher memory pressure prefer more sparse data-structures. TPUs seem to favor an implementation based on sorting intrinsics. Selective spike dropping provides a simple performance-accuracy trade-off, which could be enhanced by future autograd frameworks adapting diverging primal/tangent data-structures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。