SpikeMoE：受大脑启发的竞争路由用于灵活脉冲混合专家模型

中文摘要 AI 辅助

脉冲神经网络（SNNs）通过神经元尺度的生物启发动力学实现事件驱动计算，而混合专家模型（MoE）通过模型尺度的专家选择执行条件计算。将两者的优势相结合，有望构建灵活的神经架构。然而，一个关键挑战在于设计基于脉冲活动的专家选择机制。为解决这一问题，我们提出了一种受海马体CA1区域竞争性抑制现象启发的基于脉冲的k-WTA路由器。该路由器结合了侧向抑制和不应期，根据离散脉冲计数选择Top-K专家。在此基础上，我们提出了SpikeMoE框架，将神经元尺度的脉冲动力学与模型尺度的专家选择相集成。为应对多模态任务中不完整的多感官输入，我们进一步为SpikeMoE配备了一个两阶段缺失模态建模模块，该模块将来自观测模态池的经验原型与模态特定的可学习嵌入相结合，以构建缺失模态的表示。在视觉、语言和多模态基准上的实验表明，SpikeMoE在SNN基线中达到了最先进的性能，匹配或超过了ANN对应模型的性能，并在多种缺失模态条件下保持了鲁棒性。这些结果展示了性能与能效之间的良好权衡，验证了脉冲动力学与稀疏专家计算的集成，并凸显了SpikeMoE作为节能脑启发计算的一种有前景的方法。

英文摘要

Spiking Neural Networks (SNNs) enable event-driven computation through biologically inspired dynamics at the neuronal scale, while Mixture-of-Experts (MoE) perform conditional computation through expert selection at the model scale. Integrating their strengths offers potential for flexible neural architectures. A key challenge, however, lies in designing an expert selection mechanism based on spiking activity. To address this, we introduce a spike-based k-WTA Router inspired by competition-inhibition observed in the hippocampal CA1 region. The router incorporates lateral inhibition and refractory period to select Top-K experts according to discrete spike counts. Building on this, we present SpikeMoE, a framework that integrates neuronal-scale spiking dynamics with model-scale expert selection. To address incomplete multisensory inputs in multimodal tasks, we further equip SpikeMoE with a two-stage missing-modality modeling module that combines empirical prototypes from an observed-modality pool with modality-specific learnable embeddings to construct missing-modality representations. Experiments on vision, language, and multimodal benchmarks demonstrate that SpikeMoE achieves state-of-the-art performance among the SNN baselines, matches or exceeds the performance of ANN counterparts, and maintains robustness across diverse missing-modality conditions. These results demonstrate a favorable trade-off between performance and energy efficiency, validating the integration of spiking dynamics with sparse expert computation and highlighting SpikeMoE as a promising approach to energy-efficient brain-inspired computing.

发表机构

• University of Electronic Science and Technology of China（电子科技大学）

机构由 AI 辅助整理，请以论文原文为准。