arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.18271cs.AR
收藏

用于循环神经网络的模拟电阻交叉阵列中的高效非线性函数逼近

Efficient Nonlinear Function Approximation in Analog Resistive Crossbars for Recurrent Neural Networks

Junyi Yang, Ruibin Mao, Mingrui Jiang, Yichuan Cheng, Pao-Sheng Vincent Sun, Shuai Dong, Giacomo Pedretti, Xia Sheng, Jim Ignowski, Haoliang Li, Can Li, Arindam Basu

AI总结:

针对模拟内存计算(IMC)部署循环神经网络(RNN)时非线性激活函数带来的时间和能量损耗问题,提出一种集成斜坡模数转换(ADC)的非线性函数实现方案,利用忆阻器生成预失真斜坡电压,实验证明可提升RNN的面积、能量效率和吞吐量。

摘要:

模拟内存计算(IMC)利用物理原理在电阻存储阵列中并行计算,已证明可实现深度神经网络(DNN)中卷积层和全连接层的高能效、低延迟部署。然而,广泛应用于语音识别和自然语言处理的循环神经网络(RNN)通过该方法仅取得有限成功,这可归因于此类模型中大量存在的非线性激活函数部署时产生的显著时间和能量损耗。本研究通过实验展示了一种集成斜坡模数转换(ADC)的非线性激活函数实现方案,该方案部署于存储器外围以优化RNN的内存计算实现。我们的方法使用额外一列忆阻器生成预失真的斜坡电压,使比较器输出直接逼近目标非线性函数。我们通过实验展示了利用忆阻阵列编程不同非线性函数,并模拟将其集成到RNN中以解决关键词识别和语言建模任务。与其他方法相比,我们证明了由于内存内可编程斜坡发生器消除了数字处理开销,面积效率、能量效率和吞吐量均实现了多倍提升。

英文摘要:

Analog In-memory Computing (IMC) has demonstrated energy-efficient and low latency implementation of convolution and fully-connected layers in deep neural networks (DNN) by using physics for computing in parallel resistive memory arrays. However, recurrent neural networks (RNN) that are widely used for speech-recognition and natural language processing have tasted limited success with this approach. This can be attributed to the significant time and energy penalties incurred in implementing nonlinear activation functions that are abundant in such models. In this work, we experimentally demonstrate the implementation of a non-linear activation function integrated with a ramp analog-to-digital conversion (ADC) at the periphery of the memory to improve in-memory implementation of RNNs. Our approach uses an extra column of memristors to produce an appropriately pre-distorted ramp voltage such that the comparator output directly approximates the desired nonlinear function. We experimentally demonstrate programming different nonlinear functions using a memristive array and simulate its incorporation in RNNs to solve keyword spotting and language modelling tasks. Compared to other approaches, we demonstrate manifold increase in area-efficiency, energy-efficiency and throughput due to the in-memory, programmable ramp generator that removes digital processing overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑