Spiking Neural Networks (SNNs), with their brain-inspired spatiotemporal dynamics and spike-driven computation, have emerged as promising energy-efficient alternatives to Artificial Neural Networks (ANNs). However, existing SNNs typically replicate inputs directly or aggregate them into frames at fixed intervals. Such strategies lead to neurons receiving nearly identical stimuli across time steps, severely limiting the model's expressive power, particularly in complex tasks like object detection. In this work, we propose the Temporal Dynamics Enhancer (TDE) to strengthen SNNs' capacity for temporal information modeling. TDE consists of two modules: a Spiking Encoder (SE) that generates diverse input stimuli across time steps, and an Attention Gating Module (AGM) that guides the SE generation based on inter-temporal dependencies. Moreover, to eliminate the high-energy multiplication operations introduced by the AGM, we propose a Spike-Driven Attention (SDA) to reduce attention-related energy consumption. Extensive experiments demonstrate that TDE can be seamlessly integrated into existing SNN-based detectors and consistently outperforms state-of-the-art methods, achieving mAP50-95 scores of 57.7% on the static PASCAL VOC dataset and 47.6% on the neuromorphic EvDET200K dataset. In terms of energy consumption, the SDA consumes only 0.240 times the energy of conventional attention modules.
Vertebrate retinas are highly-efficient in processing trivial visual tasks such as detecting moving objects, yet a complex challenges for modern computers. In vertebrates, the detection of object motion is performed by specialised retinal cells named Object Motion Sensitive Ganglion Cells (OMS-GC). OMS-GC process continuous visual signals and generate spike patterns that are post-processed by the Visual Cortex. Our previous Hybrid Sensitive Motion Detector (HSMD) algorithm was the first hybrid algorithm to enhance Background subtraction (BS) algorithms with a customised 3-layer Spiking Neural …
This paper proposes a Fully Spiking Hybrid Neural Network (FSHNN) for energy-efficient and robust object detection in resource-constrained platforms. The network architecture is based on Convolutional SNN using leaky-integrate-fire neuron models. The model combines unsupervised Spike Time-Dependent Plasticity (STDP) learning with back-propagation (STBP) learning methods and also uses Monte Carlo Dropout to get an estimate of the uncertainty error. FSHNN provides better accuracy compared to DNN based object detectors while being 150X energy-efficient. It also outperforms these object detectors,…
SpikeDet: Better Firing Patterns for Accurate and Energy-Efficient Object Detection with Spiking Neural Networks
School of Microelectronics（微电子学院）；School of Electrical and Information Engineering（电气与信息工程学院）；Optoelectronic Thin Film Device and Technology Research Institute（光电薄膜器件与技术研究所）
The detection of moving objects is a trivial task performed by vertebrate retinas, yet a complex computer vision task. Object-motion-sensitive ganglion cells (OMS-GC) are specialised cells in the retina that sense moving objects. OMS-GC take as input continuous signals and produce spike patterns as output, that are transmitted to the Visual Cortex via the optic nerve. The Hybrid Sensitive Motion Detector (HSMD) algorithm proposed in this work enhances the GSOC dynamic background subtraction (DBS) algorithm with a customised 3-layer spiking neural network (SNN) that outputs spiking responses ak…
Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Tongyi Lab, Alibaba Group（阿里巴巴集团通义实验室）；Peng Cheng Laboratory（鹏城实验室）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
International Digital Economy Academy (IDEA)（国际数字经济学院（IDEA））；Institute for AI, Tsinghua-Bosch Joint Center for ML, Tsinghua University（清华大学人工智能研究院、清华-博世联合机器学习中心）；The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Microsoft Research（微软研究院）；South China University of Technology（华南理工大学）
Vmem-$φ$: Low-Compute Out-of-Distribution Detection in Spiking Neural Networks from Membrane-Potential Statistics
Indian Institute of Technology Kharagpur（印度理工学院卡哈拉格普尔分校）；Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；University of Toronto（多伦多大学）
University of Tehran（德黑兰大学）；CNRS – Université Toulouse 3（法国国家科学研究中心-图卢兹第三大学）
Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-propagation - that having many layers increases the recognition robustness, an approach known as deep learning. We thus designed a deep SNN, comprising several convolutional (trainable with STDP) and p…
Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；Peng Cheng Laboratory（鹏城实验室）；Beijing Institute of Technology（北京理工大学）；School of Computer Science, Peking University（北京大学计算机学院）；School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University（北京大学深圳研究生院电子与计算机工程学院）
