Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and energy efficient implementations in neuromorphic hardware. However the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We design a spiking version of the successful residual network architecture and provide an in-depth study on the possible implementations of spiking residual connections. This study shows how, depending on the use case, the optimal residual connection implementation may vary. Additionally, we empirically compare different techniques in image classification datasets taken from the best performing networks. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in CIFAR-10 (94.14%) and CIFAR-100 (74.65%) datasets and matches the state of the art in DVS-CIFAR10 (72.98%), with less parameters than the previous state of the art and without the need for ANN-SNN conversion. Code available at https://github.com/VicenteAlex/Spiking_ResNet
Spiking neural networks (SNNs) have received significant attention for their biological plausibility. SNNs theoretically have at least the same computational power as traditional artificial neural networks (ANNs). They possess potential of achieving energy-efficiency while keeping comparable performance to deep neural networks (DNNs). However, it is still a big challenge to train a very deep SNN. In this paper, we propose an efficient approach to build a spiking version of deep residual network (ResNet). ResNet is considered as a kind of the state-of-the-art convolutional neural networks (CNNs…
Spiking Neural Networks (SNNs) have recently attracted significant research interest as the third generation of artificial neural networks that can enable low-power event-driven data analytics. The best performing SNNs for image recognition tasks are obtained by converting a trained Analog Neural Network (ANN), consisting of Rectified Linear Units (ReLU), to SNN composed of integrate-and-fire neurons with "proper" firing thresholds. The converted SNNs typically incur loss in accuracy compared to that provided by the original ANN and require sizable number of inference time-steps to achieve the…
Biologically inspired spiking neural networks (SNNs) are widely used to realize ultralow-power energy consumption. However, deep SNNs are not easy to train due to the excessive firing of spiking neurons in the hidden layers. To tackle this problem, we propose a novel but simple normalization technique called postsynaptic potential normalization. This normalization removes the subtraction term from the standard normalization and uses the second raw moment instead of the variance as the division term. The spike firing can be controlled, enabling the training to proceed appropriating, by conducti…
Tsinghua University（清华大学）；Technische Universität Graz（格拉茨工业大学）；Xi’an Jiaotong University（西安交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Despite the rapid progress of neuromorphic computing, inadequate capacity and insufficient representation power of spiking neural networks (SNNs) severely restrict their application scope in practice. Residual learning and shortcuts have been evidenced as an important approach for training deep neural networks, but rarely did previous work assess their applicability to the characteristics of spike-based communication and spatiotemporal dynamics. In this paper, we first identify that this negligence leads to impeded information flow and the accompanying degradation problem in previous residual …
SCNO: Spiking Compositional Neural Operator -- Towards a Neuromorphic Foundation Model for Nuclear PDE Solving
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Indian Institute of Technology Delhi（印度理工学院德里分校）；National Center for Supercomputing Applications（国家超级计算应用中心）
Towards Lossless ANN-SNN Conversion under Ultra-Low Latency with Dual-Phase Optimization
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Zhejiang Lab（之江实验室）；School of Integrated Circuits, Peking University（北京大学集成电路学院）；College of Computer Science, Zhejiang University of Technology（浙江工业大学计算机学院）；State Key Laboratory of Brain-Machine Intelligence, Zhejiang University（浙江大学脑机智能国家重点实验室）
Institute of Information and Communication Technology, Bangladesh University of Engineering and Technology（孟加拉工程技术大学信息与通信技术学院）
Breast cancer is a common fatal disease for women. Early diagnosis and detection is necessary in order to improve the prognosis of breast cancer affected people. For predicting breast cancer, several automated systems are already developed using different medical imaging modalities. This paper provides a systematic review of the literature on artificial neural network (ANN) based models for the diagnosis of breast cancer via mammography. The advantages and limitations of different ANN models including spiking neural network (SNN), deep belief network (DBN), convolutional neural network (CNN), …
DBHN-Net: Dual-Branch Hybrid Neural Network For Low-Complexity Monaural Speech Enhancement
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, (School of Computer Science and Technology), Anhui University（光电信息获取与防护技术国家重点实验室（计算机科学与技术学院），安徽大学）；China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd（中国电信人工智能技术（北京）有限公司）；Institute of Acoustics, University of Chinese Academy of Sciences（中国科学院声学研究所）；Institute of Artificial Intelligence (TeleAI), China Telecom, China（人工智能研究所（TeleAI），中国电信，中国）
