arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.04588cs.ET
收藏

利用无源RRAM纵横阵列的长短期记忆器实现

Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array

Honey Nikam, Siddharth Satyam, Shubham Sahay

摘要:

在这个大数据时代,对跨序列数据提取时间相关性和执行基于上下文的学习的需求不断增长,导致了长短期记忆(LSTM)网络的发展。此外,迫切需要在能量有限的紧凑型物联网(IoT)边缘设备上执行这些时间序列数据相关的应用,包括语音/视频处理和识别,语言建模和翻译等。为此,在这项工作中,我们首次提出了一种利用无源电阻式随机存取存储器(RRAM)交叉阵列的面积和能源效率极高的LSTM网络实现。我们开发了一个硬件感知的LSTM网络仿真框架,并对所提出的LSTM实现进行了广泛的分析,考虑到非理想的硬件伪影,如空间(设备到设备)和时间变化,非线性,噪声等,利用实验校准的无源RRAM交叉阵列的综合现象学模型。我们的研究结果表明,所提出的基于无源RRAM交叉杆的LSTM网络实现不仅在面积方面比现有的数字和基于有源1 T-1 R交叉杆的LSTM实现高出三个数量级以上,在训练能量方面高出两个数量级,而且对空间和时间变化以及噪声具有鲁棒性,收敛速度更快。我们的工作可能为在无源RRAM交叉阵列上实验实现LSTM网络提供激励。

英文摘要:

The ever-increasing demand to extract temporal correlations across sequential data and perform context-based learning in this era of big data has led to the development of long short-term memory (LSTM) networks. Furthermore, there is an urgent need to perform these time-series data-dependent applications including speech/video processing and recognition, language modelling and translation, etc. on compact internet-of-things (IoT) edge devices with limited energy. To this end, in this work, for the first time, we propose an extremely area- and energy-efficient LSTM network implementation exploiting the passive resistive random access memory (RRAM) crossbar array. We developed a hardware-aware LSTM network simulation framework and performed an extensive analysis of the proposed LSTM implementation considering the non-ideal hardware artifacts such as spatial (device-to-device) and temporal variations, non-linearity, noise, etc. utilizing an experimentally calibrated comprehensive phenomenological model for passive RRAM crossbar array. Our results indicate that the proposed passive RRAM crossbar-based LSTM network implementation not only outperforms the prior digital and active 1T-1R crossbar-based LSTM implementations by more than three orders of magnitude in terms of area and two orders of magnitude in terms of training energy for identical network accuracy, but also exhibits robustness against spatial and temporal variations and noise, and a faster convergence rate. Our work may provide the incentive for experimental realization of LSTM networks on passive RRAM crossbar arrays.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑