arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.13672cs.CV
收藏

Q-SNNs:量化脉冲神经网络

Q-SNNs: Quantized Spiking Neural Networks

University of Electronic Science and Technology of China(电子科技大学);Northumbria University(诺森比亚大学)

Wenjie Wei, Yu Liang, Ammar Belatreche, Yichen Xiao, Honglin Cao, Zhenbang Ren, Guoqing Wang, Malu Zhang, Yang Yang

AI总结:

本文提出量化脉冲神经网络Q-SNN,对突触权重和膜电位进行量化压缩以降低内存与计算开销,并设计基于信息熵的权重-脉冲双重调节方法WS-DR来抑制性能下降,在多种数据集上同时实现了更小的模型规模和更高的精度。

摘要:

受脑启发的脉冲神经网络(SNNs)利用稀疏脉冲来表示信息,并以异步事件驱动的方式进行处理,为下一代机器智能提供了一种高能效的范式。然而,当前SNN领域的研究重点是通过开发大规模模型来优化精度,这限制了它们在资源受限和低功耗边缘设备上的可行性。为应对这一挑战,我们提出了一种轻量级且硬件友好的量化脉冲神经网络(Q-SNN),该网络对突触权重和膜电位同时应用量化。通过显著压缩这两个关键要素,所提出的Q-SNNs大幅降低了内存使用和计算复杂度。此外,为防止这种压缩导致的性能下降,我们提出了一种受信息熵理论启发的全新权重-脉冲双重调节(WS-DR)方法。在包括静态数据集和神经形态数据集在内的多种数据集上的实验评估表明,我们的Q-SNNs在模型规模和精度方面均优于现有方法。这些在效率和效能方面达到最先进水平的结果表明,所提出的方法能够显著提升边缘智能计算。

英文摘要:

Brain-inspired Spiking Neural Networks (SNNs) leverage sparse spikes to represent information and process them in an asynchronous event-driven manner, offering an energy-efficient paradigm for the next generation of machine intelligence. However, the current focus within the SNN community prioritizes accuracy optimization through the development of large-scale models, limiting their viability in resource-constrained and low-power edge devices. To address this challenge, we introduce a lightweight and hardware-friendly Quantized SNN (Q-SNN) that applies quantization to both synaptic weights and membrane potentials. By significantly compressing these two key elements, the proposed Q-SNNs substantially reduce both memory usage and computational complexity. Moreover, to prevent the performance degradation caused by this compression, we present a new Weight-Spike Dual Regulation (WS-DR) method inspired by information entropy theory. Experimental evaluations on various datasets, including static and neuromorphic, demonstrate that our Q-SNNs outperform existing methods in terms of both model size and accuracy. These state-of-the-art results in efficiency and efficacy suggest that the proposed method can significantly improve edge intelligent computing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑