首页/论文/2002.11843

arXiv 2002.118432020-02cs.NEcs.LGeess.IVq-bio.NC

A Deep Unsupervised Feature Learning Spiking Neural Network with Binarized Classification Layers for EMNIST Classification using SpykeFlow

Boise State University（博伊西州立大学）；University of Delaware（特拉华大学）

Ruthvik Vaila, John Chiasson, Vishal Saxena

备注：: A section of of this work is Submitted to IEEE TETCI 2020 Journal

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

End user AI is trained on large server farms with data collected from the users. With ever increasing demand for IOT devices, there is a need for deep learning approaches that can be implemented (at the edge) in an energy efficient manner. In this work we approach this using spiking neural networks. The unsupervised learning technique of spike timing dependent plasticity (STDP) using binary activations are used to extract features from spiking input data. Gradient descent (backpropagation) is used only on the output layer to perform the training for classification. The accuracies obtained for the balanced EMNIST data set compare favorably with other approaches. The effect of stochastic gradient descent (SGD) approximations on learning capabilities of our network are also explored.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

The fine line between dead neurons and sparsity in binarized spiking neural networks

University of Michigan（密歇根大学）；University of Western Australia（西澳大利亚大学）

Spiking neural networks can compensate for quantization error by encoding information either in the temporal domain, or by processing discretized quantities in hidden states of higher precision. In theory, a wide dynamic range state-space enables multiple binarized inputs to be accumulated together, thus improving the representational capacity of individual neurons. This may be achieved by increasing the firing threshold, but make it too high and sparse spike activity turns into no spike emission. In this paper, we propose the use of `threshold annealing' as a warm-up method for firing thresho…

2022-01-28cs.LGcs.NEq-bio.NC

BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning

Shahid Beheshti University（沙希德·贝赫什提大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；CerCo UMR 5549, CNRS Université Toulouse 3（法国国家科学研究中心图卢兹第三大学 CerCo UMR 5549 实验室）

We recently proposed the S4NN algorithm, essentially an adaptation of backpropagation to multilayer spiking neural networks that use simple non-leaky integrate-and-fire neurons and a form of temporal coding known as time-to-first-spike coding. With this coding scheme, neurons fire at most once per stimulus, but the firing order carries information. Here, we introduce BS4NN, a modification of S4NN in which the synaptic weights are constrained to be binary (+1 or -1), in order to decrease memory (ideally, one bit per synapse) and computation footprints. This was done using two sets of weights: f…

2021-12-06cs.CVcs.NE

Ultra-low Latency Adaptive Local Binary Spiking Neural Network with Accuracy Loss Estimator

Spiking neural network (SNN) is a brain-inspired model which has more spatio-temporal information processing capacity and computational energy efficiency. However, with the increasing depth of SNNs, the memory problem caused by the weights of SNNs has gradually attracted attention. Inspired by Artificial Neural Networks (ANNs) quantization technology, binarized SNN (BSNN) is introduced to solve the memory problem. Due to the lack of suitable learning algorithms, BSNN is usually obtained by ANN-to-SNN conversion, whose accuracy will be limited by the trained ANNs. In this paper, we propose an u…

2022-07-31cs.NE

Algorithm and Hardware Design of Discrete-Time Spiking Neural Networks Based on Back Propagation with Binary Activations

We present a new back propagation based training algorithm for discrete-time spiking neural networks (SNN). Inspired by recent deep learning algorithms on binarized neural networks, binary activation with a straight-through gradient estimator is used to model the leaky integrate-fire spiking neuron, overcoming the difficulty in training SNNs using back propagation. Two SNN training algorithms are proposed: (1) SNN with discontinuous integration, which is suitable for rate-coded input spikes, and (2) SNN with continuous integration, which is more general and can handle input spikes with tempora…

2017-09-19cs.NE

基于柱状脉冲神经网络的持续学习

Continual Learning with Columnar Spiking Neural Networks

Chuvash State University（楚瓦什州大学）；Trapeznikov Institute of Control Sciences（特拉普琴尼科夫控制科学研究院）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）

本文提出基于CoLaNET的柱状脉冲神经网络，利用局部学习规则实现持续学习，在Permuted MNIST和MNIST/EMNIST基准上有效缓解灾难性遗忘，仅4%性能下降。

2025-08-16cs.AIcs.NE

High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning

Sun Yat-sen University（中山大学）；The University of Sydney（悉尼大学）

Spiking Neural Networks (SNNs) are brain-inspired, event-driven machine learning algorithms that have been widely recognized in producing ultra-high-energy-efficient hardware. Among existing SNNs, unsupervised SNNs based on synaptic plasticity, especially Spike-Timing-Dependent Plasticity (STDP), are considered to have great potential in imitating the learning process of the biological brain. Nevertheless, the existing STDP-based SNNs have limitations in constrained learning capability and/or slow learning speed. Most STDP-based SNNs adopted a slow-learning Fully-Connected (FC) architecture an…

2021-03-09cs.LGcs.NEq-bio.NC

更多相关论文收起更多论文

Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks

Texas A&M University（得克萨斯农工大学）

Spiking neural networks (SNNs) are positioned to enable spatio-temporal information processing and ultra-low power event-driven neuromorphic hardware. However, SNNs are yet to reach the same performances of conventional deep artificial neural networks (ANNs), a long-standing challenge due to complex dynamics and non-differentiable spike events encountered in training. The existing SNN error backpropagation (BP) methods are limited in terms of scalability, lack of proper handling of spiking discontinuities, and/or mismatch between the rate-coded loss function and computed gradient. We present a…

2019-01-19cs.LGcs.NE

脉冲异构图注意力网络

Spiking Heterogeneous Graph Attention Networks

SpikingHAN通过融合脉冲神经网络的节能特性，降低异构图学习的计算成本，同时保持性能。

2025-12-31cs.LGcs.NEcs.SI

图值得1位脉冲：当图对比学习遇上脉冲神经网络

A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks

Sun Yat-sen University（中山大学）；Ant Group（蚂蚁集团）；Nanyang Technological University（南洋理工大学）

提出SpikeGCL框架，结合脉冲神经网络实现图对比学习的1位二值化表示，在近32倍存储压缩下保持与全精度方法相当的性能，平衡了效率与表现。

2024-02-19cs.AIcs.LGcs.NE

基于LIF神经元的脉冲神经网络FPGA能量感知实现

Energy-Aware FPGA Implementation of Spiking Neural Network with LIF Neurons

Johns Hopkins University（约翰斯·霍普金斯大学）

本文提出一种基于一阶LIF神经元模型的SNN架构及硬件友好设计，在FPGA上实现，用于TinyML视觉任务，相比BCNN基线能效提升86%。

2024-11-03cs.ARcs.LGcs.NE

Exploiting High Performance Spiking Neural Networks with Efficient Spiking Patterns

CASIA（中国科学院自动化研究所）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）；University of Chinese Academy of Sciences（中国科学院大学）

Spiking Neural Networks (SNNs) use discrete spike sequences to transmit information, which significantly mimics the information transmission of the brain. Although this binarized form of representation dramatically enhances the energy efficiency and robustness of SNNs, it also leaves a large gap between the performance of SNNs and Artificial Neural Networks based on real values. There are many different spike patterns in the brain, and the dynamic synergy of these spike patterns greatly enriches the representation capability. Inspired by spike patterns in biological neurons, this paper introdu…

2023-01-29cs.CVcs.NE

SPA: Stochastic Probability Adjustment for System Balance of Unsupervised SNNs

Sun Yat-sen University（中山大学）

Spiking neural networks (SNNs) receive widespread attention because of their low-power hardware characteristic and brain-like signal response mechanism, but currently, the performance of SNNs is still behind Artificial Neural Networks (ANNs). We build an information theory-inspired system called Stochastic Probability Adjustment (SPA) system to reduce this gap. The SPA maps the synapses and neurons of SNNs into a probability space where a neuron and all connected pre-synapses are represented by a cluster. The movement of synaptic transmitter between different clusters is modeled as a Brownian-…

2021-05-06cs.LGcs.NEq-bio.NC

MT-SNN：利用多阈值增强脉冲神经网络

MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds

Beijing University of Technology（北京工业大学）

本文提出多阈值（MT）方法（含并行与级联模式）以缓解脉冲神经网络二值化导致的精度损失，在保持无乘法高能效优势的同时显著提升准确率，并在多个数据集上以更少时间步长超越现有SOTA。

2024-10-14cs.AIcs.CVcs.NE

Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication

Sandia National Laboratories（桑迪亚国家实验室）；Center for Computing Research（计算研究中心）

This paper presents a new technique for training networks for low-precision communication. Targeting minimal communication between nodes not only enables the use of emerging spiking neuromorphic platforms, but may additionally streamline processing conventionally. Low-power and embedded neuromorphic processors potentially offer dramatic performance-per-Watt improvements over traditional von Neumann processors, however programming these brain-inspired platforms generally requires platform-specific expertise which limits their applicability. To date, the majority of artificial neural networks ha…

2018-10-26cs.LGcs.NE