基于参数脆弱性加固与剪枝的CNN高性价比容错方法
Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning
- Tallinn University of Technology（塔林理工大学）
- Mälardalen University（梅拉达伦大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对CNN传统硬件容错方法开销大的问题，提出模型级参数脆弱性加固与剪枝技术，容错能力接近TMR且开销更低，剪枝后加固网络速度最高提升24%、精度损失可忽略。
AI中文摘要：
卷积神经网络（CNN）已成为安全关键应用中不可或缺的组成部分，因此其容错性引发了广泛关注。传统的依赖硬件的容错方法，如三模冗余（TMR），计算成本高昂，会给CNN带来显著的开销。容错技术既可以在硬件层面也可以在模型层面应用，而后者在不牺牲通用性的前提下提供了更高的灵活性。本文提出了一种CNN的模型级加固方法，将纠错功能直接集成到神经网络中。该方法与硬件无关，无需对底层加速器设备做任何改动。通过分析参数的脆弱性，可以选择性地复制滤波器/神经元，从而借助高效且鲁棒的校正层有效校正其输出通道。实验表明，所提方法的容错能力几乎与基于TMR的校正相当，但开销大幅降低。不过，相较于基线CNN，该方法仍存在固有开销。为解决这一问题，本文提出了一种基于参数脆弱性的高性价比剪枝技术，其性能优于传统剪枝方法，能生成规模更小的网络，且精度损失可忽略不计。值得注意的是，经过加固的剪枝CNN比未剪枝的加固CNN运行速度最高快24%。
英文摘要：
Convolutional Neural Networks (CNNs) have become integral in safety-critical applications, thus raising concerns about their fault tolerance. Conventional hardware-dependent fault tolerance methods, such as Triple Modular Redundancy (TMR), are computationally expensive, imposing a remarkable overhead on CNNs. Whereas fault tolerance techniques can be applied either at the hardware level or at the model levels, the latter provides more flexibility without sacrificing generality. This paper introduces a model-level hardening approach for CNNs by integrating error correction directly into the neural networks. The approach is hardware-agnostic and does not require any changes to the underlying accelerator device. Analyzing the vulnerability of parameters enables the duplication of selective filters/neurons so that their output channels are effectively corrected with an efficient and robust correction layer. The proposed method demonstrates fault resilience nearly equivalent to TMR-based correction but with significantly reduced overhead. Nevertheless, there exists an inherent overhead to the baseline CNNs. To tackle this issue, a cost-effective parameter vulnerability based pruning technique is proposed that outperforms the conventional pruning method, yielding smaller networks with a negligible accuracy loss. Remarkably, the hardened pruned CNNs perform up to 24\% faster than the hardened un-pruned ones.
