面向随机噪声下鲁棒 DNN 推理的 ReRAM 交叉阵列多目标优化
Multi-Objective Optimization of ReRAM Crossbars for Robust DNN Inferencing under Stochastic Noise
AI总结:
提出 ReSNA 噪声感知训练与 CF-MESMO 连续保真度多目标优化方法,以低成本获得 ReRAM 加速器在精度、面积、时间和能耗间的高质量 Pareto 前沿。
摘要:
阻变随机存取存储器(ReRAM)是设计用于深度神经网络(DNN)推理的硬件加速器的一种有前景技术。然而,ReRAM 交叉阵列中的随机噪声会降低 DNN 推理精度。我们提出了一种高性能、面积和能效高效的基于 ReRAM 的硬件加速器的设计与优化,以在存在随机噪声的情况下实现鲁棒的 DNN 推理。我们作出两项关键技术贡献。首先,我们提出一种随机噪声感知训练方法,称为 ReSNA,用于提高在具有随机噪声的 ReRAM 交叉阵列上进行 DNN 推理的精度。其次,我们提出一种信息论算法,称为 CF-MESMO,用于识别 Pareto 解集,以在多个目标之间进行权衡,包括推理精度、面积开销、执行时间和能耗。在此背景下的主要挑战是,执行 ReSNA 方法来评估每个候选 ReRAM 设计的成本高得令人难以承受。为应对这一挑战,我们通过改变训练 epoch 数量,在精度和成本之间进行权衡,对与高得令人难以承受的计算成本相关的 ReRAM 设计采用连续保真度评估。CF-MESMO 迭代选择候选 ReRAM 设计与保真度组合,使关于最优 Pareto 前沿的单位计算成本所获得的信息最大化。我们在基准 DNN 上的实验表明,所提出的算法能够高效揭示高质量 Pareto 前沿。平均而言,相对于基线配置,ReSNA 在 CIFAR-10 数据集上使 ResNet20 的推理精度提升 2.57%。此外,与流行的多目标优化算法 NSGA-II 相比,CF-MESMO 算法在达到 NSGA-II 的最佳解时实现了 90.91% 的计算成本降低。
英文摘要:
Resistive random-access memory (ReRAM) is a promising technology for designing hardware accelerators for deep neural network (DNN) inferencing. However, stochastic noise in ReRAM crossbars can degrade the DNN inferencing accuracy. We propose the design and optimization of a high-performance, area-and energy-efficient ReRAM-based hardware accelerator to achieve robust DNN inferencing in the presence of stochastic noise. We make two key technical contributions. First, we propose a stochastic-noise-aware training method, referred to as ReSNA, to improve the accuracy of DNN inferencing on ReRAM crossbars with stochastic noise. Second, we propose an information-theoretic algorithm, referred to as CF-MESMO, to identify the Pareto set of solutions to trade-off multiple objectives, including inferencing accuracy, area overhead, execution time, and energy consumption. The main challenge in this context is that executing the ReSNA method to evaluate each candidate ReRAM design is prohibitive. To address this challenge, we utilize the continuous-fidelity evaluation of ReRAM designs associated with prohibitive high computation cost by varying the number of training epochs to trade-off accuracy and cost. CF-MESMO iteratively selects the candidate ReRAM design and fidelity pair that maximizes the information gained per unit computation cost about the optimal Pareto front. Our experiments on benchmark DNNs show that the proposed algorithms efficiently uncover high-quality Pareto fronts. On average, ReSNA achieves 2.57% inferencing accuracy improvement for ResNet20 on the CIFAR-10 dataset with respect to the baseline configuration. Moreover, CF-MESMO algorithm achieves 90.91% reduction in computation cost compared to the popular multi-objective optimization algorithm NSGA-II to reach the best solution from NSGA-II.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。