基于电阻式RAM的二值化神经网络的可解释比特容错性
Towards Explainable Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks
摘要：
非易失性存储器，如电阻式RAM（RRAM），是一种新兴的节能存储，尤其适用于边缘的低功耗机器学习模型。然而，据报道，RRAM的位出错率在超低功率设置中可以高达3.3%，这对于许多用例来说可能是至关重要的。二进制神经网络（BNN）是神经网络（NN）的一种资源有效的变体，它可以容忍一定比例的错误而不损失准确性，并且在计算和存储方面需要更少的资源。BNN中的比特差错容限（BET）可以通过在训练期间翻转权重符号来实现，如Hirtzlin等人所提出的，但是他们的方法有一个明显的缺点，特别是对于全连接神经网络（FCNN）：FCNN过拟合训练中使用的错误率，这导致在较低出错率下的低准确度。此外，没有研究BET的基本原理。在这项工作中，我们改进了BNN的BET训练，旨在解释这一属性。我们提出了直通梯度近似，以改善权重符号翻转训练，通过该BNN适应比特出错率。为了解释所实现的鲁棒性，我们定义了一个度量，其目的是在没有故障注入的情况下测量BET。我们评估的度量，并发现它与准确度超过出错率的所有FCNN测试。最后，我们探讨了一种新的正则化优化相对于这个指标的影响，其目的是提供一个可配置的权衡精度和BET。
英文摘要：
Non-volatile memory, such as resistive RAM (RRAM), is an emerging energy-efficient storage, especially for low-power machine learning models on the edge. It is reported, however, that the bit error rate of RRAMs can be up to 3.3% in the ultra low-power setting, which might be crucial for many use cases. Binary neural networks (BNNs), a resource efficient variant of neural networks (NNs), can tolerate a certain percentage of errors without a loss in accuracy and demand lower resources in computation and storage. The bit error tolerance (BET) in BNNs can be achieved by flipping the weight signs during training, as proposed by Hirtzlin et al., but their method has a significant drawback, especially for fully connected neural networks (FCNN): The FCNNs overfit to the error rate used in training, which leads to low accuracy under lower error rates. In addition, the underlying principles of BET are not investigated. In this work, we improve the training for BET of BNNs and aim to explain this property. We propose straight-through gradient approximation to improve the weight-sign-flip training, by which BNNs adapt less to the bit error rates. To explain the achieved robustness, we define a metric that aims to measure BET without fault injection. We evaluate the metric and find that it correlates with accuracy over error rate for all FCNNs tested. Finally, we explore the influence of a novel regularizer that optimizes with respect to this metric, with the aim of providing a configurable trade-off in accuracy and BET.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
