损失塑造增强了脉冲神经网络中基于Eventprop的精确梯度学习
Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks
AI总结：
本文扩展Eventprop算法以支持更广泛的损失函数，结合数据增强、正则化和延迟线输入训练循环脉冲神经网络，在多个语音数据集上达到SOTA，且比代理梯度法快3倍、省4倍内存。
摘要：
基于事件的机器学习有望在未来神经形态硬件上实现更高能效的AI。在此，我们研究如何将近期发现的Eventprop算法（用于脉冲神经网络中精确梯度下降）扩展到具有挑战性的关键词识别基准。我们在GPU-enhanced Neural Networks框架中实现了Eventprop，并用它在Spiking Heidelberg Digits和Spiking Speech Commands数据集上训练循环脉冲神经网络。我们发现学习过程强烈依赖于损失函数，因此将Eventprop扩展到更广泛的损失函数类以实现有效训练。随后，我们测试了大量数据增强和正则化方法，并探索了不同的网络结构，以及异构和可训练的时间尺度。我们发现，当结合两种特定的数据增强、适当的正则化和延迟线输入时，具有一个循环层的Eventprop网络在Spiking Heidelberg Digits上达到了最先进的性能，并在Spiking Speech Commands上取得了良好的准确率。与一种领先的基于代理梯度的SNN训练方法相比，我们的GeNN Eventprop实现速度快3倍，内存使用少4倍。这项工作是迈向当前机器学习范式的低功耗神经形态替代方案的重要一步。
英文摘要：
Event-based machine learning promises more energy-efficient AI on future neuromorphic hardware. Here, we investigate how the recently discovered Eventprop algorithm for gradient descent on exact gradients in spiking neural networks can be scaled up to challenging keyword recognition benchmarks. We implemented Eventprop in the GPU-enhanced Neural Networks framework and used it for training recurrent spiking neural networks on the Spiking Heidelberg Digits and Spiking Speech Commands datasets. We found that learning depended strongly on the loss function and extended Eventprop to a wider class of loss functions to enable effective training. We then tested a large number of data augmentations and regularisations as well as exploring different network structures; and heterogeneous and trainable timescales. We found that when combined with two specific augmentations, the right regularisation and a delay line input, Eventprop networks with one recurrent layer achieved state-of-the-art performance on Spiking Heidelberg Digits and good accuracy on Spiking Speech Commands. In comparison to a leading surrogate-gradient-based SNN training method, our GeNN Eventprop implementation is 3X faster and uses 4X less memory. This work is a significant step towards a low-power neuromorphic alternative to current machine learning paradigms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
