一点能量就能走很长的路：从卷积神经网络构建节能、准确的脉冲神经网络
A Little Energy Goes a Long Way: Build an Energy-Efficient, Accurate Spiking Neural Network from Convolutional Neural Network
摘要：
脉冲神经网络（SNN）提供了处理空间-时间数据（或者换句话说，真实世界的感觉数据）的固有能力，但是遭受训练高精度模型的困难。SNN研究的一个主要思路是将预训练的卷积神经网络（CNN）转换为相同结构的SNN。最先进的转换方法正在接近精度极限，即，SNN相对于原始CNN的接近零的准确性损失。然而，我们注意到，只有当处理输入消耗更多的能量时，这才成为可能。在本文中，我们认为，这种趋势的“能源的准确性”是没有必要的-一点点的能源可以走很长的路，以实现近零的准确性损失。具体而言，我们提出了一种新的CNN到SNN转换方法，该方法能够使用合理短的尖峰训练（例如，CIFAR 10图像的256个时间步），以实现接近零的精度损失。新的转换方法，命名为显式电流控制（ECC），包含三种技术（电流归一化，残留消除阈值，和批量归一化的一致性维护），以显式地控制电流流过SNN处理输入时。我们将ECC实现到一个昵称为SpKeras的工具中，该工具可以方便地导入Keras CNN模型并将其转换为SNN。我们使用该工具进行了一系列广泛的实验-使用VGG 16和各种数据集，如CIFAR 10和CIFAR 100-并与最先进的转换方法进行比较。结果表明，ECC是一种很有前途的方法，可以同时优化能量消耗和准确性损失。
英文摘要：
Spiking neural networks (SNNs) offer an inherent ability to process spatial-temporal data, or in other words, realworld sensory data, but suffer from the difficulty of training high accuracy models. A major thread of research on SNNs is on converting a pre-trained convolutional neural network (CNN) to an SNN of the same structure. State-of-the-art conversion methods are approaching the accuracy limit, i.e., the near-zero accuracy loss of SNN against the original CNN. However, we note that this is made possible only when significantly more energy is consumed to process an input. In this paper, we argue that this trend of "energy for accuracy" is not necessary -- a little energy can go a long way to achieve the near-zero accuracy loss. Specifically, we propose a novel CNN-to-SNN conversion method that is able to use a reasonably short spike train (e.g., 256 timesteps for CIFAR10 images) to achieve the near-zero accuracy loss. The new conversion method, named as explicit current control (ECC), contains three techniques (current normalisation, thresholding for residual elimination, and consistency maintenance for batch-normalisation), in order to explicitly control the currents flowing through the SNN when processing inputs. We implement ECC into a tool nicknamed SpKeras, which can conveniently import Keras CNN models and convert them into SNNs. We conduct an extensive set of experiments with the tool -- working with VGG16 and various datasets such as CIFAR10 and CIFAR100 -- and compare with state-of-the-art conversion methods. Results show that ECC is a promising method that can optimise over energy consumption and accuracy loss simultaneously.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
