In the training and inference of spiking neural networks (SNNs), direct training and lightweight computation methods have been orthogonally developed, aimed at reducing power consumption. However, only a limited number of approaches have applied these two mechanisms simultaneously and failed to fully leverage the advantages of SNN-based vision transformers (ViTs) since they were originally designed for convolutional neural networks (CNNs). In this paper, we propose AT-SNN designed to dynamically adjust the number of tokens processed during inference in SNN-based ViTs with direct training, wherein power consumption is proportional to the number of tokens. We first demonstrate the applicability of adaptive computation time (ACT), previously limited to RNNs and ViTs, to SNN-based ViTs, enhancing it to discard less informative spatial tokens selectively. Also, we propose a new token-merge mechanism that relies on the similarity of tokens, which further reduces the number of tokens while enhancing accuracy. We implement AT-SNN to Spikformer and show the effectiveness of AT-SNN in achieving high energy efficiency and accuracy compared to state-of-the-art approaches on the image classification tasks, CIFAR10, CIFAR-100, and TinyImageNet. For example, our approach uses up to 42.4% fewer tokens than the existing best-performing method on CIFAR-100, while conserving higher accuracy.
Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement
ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC联合学院）；College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）
Motif-topology and Reward-learning improved Spiking Neural Network for Efficient Multi-sensory Integration
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）；School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）
Noise-Robust Deep Spiking Neural Networks with Temporal Information
Seoul National University（首尔大学）；Institute of New Media and Communications, Seoul National University（首尔大学新媒体与通信研究所）；ASRI and Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔大学自动系统研究所及人工智能跨学科项目）
Recent Advances and New Frontiers in Spiking Neural Networks
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Reconstructing Spiking Neural Networks Using a Single Neuron with Autapses
Brain-Apparatus Communication Institute, University of Electronic Science and Technology of China（脑器通信研究院，电子科技大学）；School of Artificial Intelligence, Chongqing University of Education（人工智能学院，重庆教育学院）
Toward Robust Spiking Neural Network Against Adversarial Perturbation
University of California, Santa Barbara（加州大学圣巴巴拉分校）；Drexel University（德雷塞尔大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；Tsinghua University（清华大学）
ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networks
The College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Artificial Intelligence, Dalian University of Technology（大连理工大学人工智能学院）；School of Computing, University of Leeds（利兹大学计算学院）；Research Institute of Intelligent Computing, Zhejiang Lab（之江实验室智能计算研究院）
LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networks
Zhejiang University（浙江大学）；Zhejiang University School of Medicine（浙江大学医学院）；Affiliated Mental Health Center & Hangzhou Seventh People’s Hospital（附属精神卫生中心&杭州市第七人民医院）
Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization
Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所类脑智能研究中心）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）；Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）
