SWIM:用于内存计算神经加速器的选择性写验证
SWIM: Selective Write-Verify for Computing-in-Memory Neural Accelerators
摘要:
基于非易失性新兴存储器的内存计算架构由于其高能效而在深度神经网络(DNN)加速方面表现出巨大的潜力。然而,这些新兴设备在映射过程期间可能遭受显著变化,即,将权重编程到装置),且如果不加以处理,可导致显著的准确度降级。权重映射的非理想性可以通过具有写验证方案的迭代编程来补偿,即,读取电导并在必要时重写。在所有现有的作品中,这种做法被应用于DNN的每个权重,因为它正在被映射,这需要大量的编程时间。在这项工作中,我们证明了只需要选择一小部分权重进行写验证就可以保持DNN的准确性,从而实现显著的加速。我们进一步介绍了二阶导数为基础的技术SWIM,它只需要一个单一的通过正向和反向传播,有效地选择需要写验证的权重。针对不同数据集的各种DNN架构的实验结果表明,与传统的完全写入验证相比,SWIM可以实现高达10倍的编程加速,同时获得相当的准确性。
英文摘要:
Computing-in-Memory architectures based on non-volatile emerging memories have demonstrated great potential for deep neural network (DNN) acceleration thanks to their high energy efficiency. However, these emerging devices can suffer from significant variations during the mapping process i.e., programming weights to the devices), and if left undealt with, can cause significant accuracy degradation. The non-ideality of weight mapping can be compensated by iterative programming with a write-verify scheme, i.e., reading the conductance and rewriting if necessary. In all existing works, such a practice is applied to every single weight of a DNN as it is being mapped, which requires extensive programming time. In this work, we show that it is only necessary to select a small portion of the weights for write-verify to maintain the DNN accuracy, thus achieving significant speedup. We further introduce a second derivative based technique SWIM, which only requires a single pass of forward and backpropagation, to efficiently select the weights that need write-verify. Experimental results on various DNN architectures for different datasets show that SWIM can achieve up to 10x programming speedup compared with conventional full-blown write-verify while attaining a comparable accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。