面向可实时部署在边缘设备的虚拟感知的、具备蒸馏辅助的低延迟激活正则化稀疏神经算子
Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Neuromorphic Virtual Sensing
浏览论文内容
中文总结 AI 辅助
本文提出SAR层、合成知识蒸馏等方法，改进NOMAD架构与VSN，在换热器数据集上大幅降低LEE指标与L2误差，为边缘部署的能效虚拟感知提供了新框架。
中文摘要 AI 辅助
虚拟感知使数字孪生和安全关键系统能够实时重建与预测时空物理过程。然而，传统计算方法与数据驱动方法在边缘部署时往往面临泛化性、延迟及能效方面的挑战。神经算子是一种有前景的替代方案，但仍依赖高功耗硬件。脉冲神经元与神经形态计算可提升能效，但替代梯度训练与多步脉冲会带来收敛性与延迟挑战。本文提出Sparse-Activation-ReLU（SAR）层，这是一种无需替代梯度训练、可促进激活稀疏性且兼容事件驱动计算的单步替代方案。在基于主干的NOMAD架构中，SAR相比可变脉冲神经元（VSN）与泄漏积分放电（LIF）实现，在延迟-误差-能效（LEE）综合指标上实现了五倍以上的提升。本文进一步分析脉冲熵与特征使用情况，引入合成知识蒸馏，将LEE分数降低两倍以上。最后，本文通过基于ReLU的脉冲损失与图邻域阈值改进VSN。在换热器数据集上，这些方法分别将L2误差降低两倍以上与近七倍，同时减少脉冲与空间聚合。总体而言，本研究为能效虚拟感知提供了一种替代框架，可适配神经形态或其他边缘设备集成，有望成为未来基于稀疏性或受大脑启发的脉冲的高效设计在延迟、能效与误差性能方面的对比金标准。
英文摘要
Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.
发表机构
- National Center for Supercomputing Applications（国家超级计算应用中心）
机构由 AI 辅助整理，请以论文原文为准。
