脉冲神经网络的特征归因解释
Feature Attribution Explanations for Spiking Neural Networks
AI总结：
针对脉冲神经网络在关键应用中的可解释性需求，提出融合脉冲时间与模型权重、纳入缺失脉冲概念的时间脉冲归因方法，经多指标验证具备优良解释特性，为可解释脉冲神经网络研究奠定基础。
摘要：
第三代人工神经网络——脉冲神经网络（Spiking Neural Networks, SNNs）可在硬件上高效实现。它们在神经形态芯片上的落地开启了广泛的应用场景，比如基于机器学习的自主控制与智能生物医疗设备。但在关键应用中，洞悉SNNs的推理逻辑十分重要，因此需要为SNNs配备解释决策生成过程的能力。我们提出了\textit{时间脉冲归因}（Temporal Spike Attribution, TSA），一种面向SNNs的局部解释方法。为计算解释结果，我们聚合了模型内部变量中的所有可用信息：脉冲时间与模型权重。我们在人工时序数据和真实世界时序数据上评估TSA，并依据多个量化指标衡量解释质量。我们发现TSA能正确识别出与决策相关的一小部分输入特征子集（即具备输出完备性与紧凑性），且对相似输入生成相似的解释（即具备连续性）。此外，我们的实验表明，纳入“缺失脉冲”的概念可提升解释质量。本研究可作为可解释SNNs领域的起点，未来在硬件上的实现将能在广泛应用场景中同时输出预测结果与解释。源代码可访问https://github.com/ElisaNguyen/tsa-explanations获取。
英文摘要：
Third-generation artificial neural networks, Spiking Neural Networks (SNNs), can be efficiently implemented on hardware. Their implementation on neuromorphic chips opens a broad range of applications, such as machine learning-based autonomous control and intelligent biomedical devices. In critical applications, however, insight into the reasoning of SNNs is important, thus SNNs need to be equipped with the ability to explain how decisions are reached. We present \textit{Temporal Spike Attribution} (TSA), a local explanation method for SNNs. To compute the explanation, we aggregate all information available in model-internal variables: spike times and model weights. We evaluate TSA on artificial and real-world time series data and measure explanation quality w.r.t. multiple quantitative criteria. We find that TSA correctly identifies a small subset of input features relevant to the decision (i.e., is output-complete and compact) and generates similar explanations for similar inputs (i.e., is continuous). Further, our experiments show that incorporating the notion of \emph{absent} spikes improves explanation quality. Our work can serve as a starting point for explainable SNNs, with future implementations on hardware yielding not only predictions but also explanations in a broad range of application scenarios. Source code is available at https://github.com/ElisaNguyen/tsa-explanations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
