使用计算相变存储器的精确深度神经网络推理
Accurate deep neural network inference using computational phase-change memory
摘要:
存内计算是一种很有前途的非冯·诺依曼方法,用于制作节能的深度学习推理硬件。电阻式存储器设备的交叉开关阵列可以用于对网络权重进行编码,并且在没有数据的中间移动的情况下执行有效的模拟矩阵向量乘法。然而,由于器件的可变性和噪声,网络需要以特定的方式进行训练,使得将数字训练的权重传递到模拟电阻式存储器器件不会导致精度的显著损失。在这里,我们介绍了一种训练ResNet型卷积神经网络的方法,该方法在将权重传输到基于相变存储器(PCM)的存内计算硬件时不会导致明显的精度损失。我们还提出了一种补偿技术,利用批量归一化参数,以提高随着时间的推移的准确性保留。在将训练的权重映射到PCM之后,我们在CIFAR-10数据集上实现了93.7%的分类准确率,在ImageNet基准测试中达到了71.6%的top-1准确率。我们在CIFAR-10上使用ResNet-32的硬件结果表明,在一天的时间内保持了93.5%以上的准确度,其中网络的361,722个突触权重中的每一个都在差分配置中组织的两个PCM设备上编程。
英文摘要:
In-memory computing is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. Crossbar arrays of resistive memory devices can be used to encode the network weights and perform efficient analog matrix-vector multiplications without intermediate movements of data. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to in-memory computing hardware based on phase-change memory (PCM). We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on the CIFAR-10 dataset and a top-1 accuracy on the ImageNet benchmark of 71.6% after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one day period, where each of the 361,722 synaptic weights of the network is programmed on just two PCM devices organized in a differential configuration.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。