Analog In-memory Computing (IMC) has demonstrated energy-efficient and low latency implementation of convolution and fully-connected layers in deep neural networks (DNN) by using physics for computing in parallel resistive memory arrays. However, recurrent neural networks (RNN) that are widely used for speech-recognition and natural language processing have tasted limited success with this approach. This can be attributed to the significant time and energy penalties incurred in implementing nonlinear activation functions that are abundant in such models. In this work, we experimentally demonstrate the implementation of a non-linear activation function integrated with a ramp analog-to-digital conversion (ADC) at the periphery of the memory to improve in-memory implementation of RNNs. Our approach uses an extra column of memristors to produce an appropriately pre-distorted ramp voltage such that the comparator output directly approximates the desired nonlinear function. We experimentally demonstrate programming different nonlinear functions using a memristive array and simulate its incorporation in RNNs to solve keyword spotting and language modelling tasks. Compared to other approaches, we demonstrate manifold increase in area-efficiency, energy-efficiency and throughput due to the in-memory, programmable ramp generator that removes digital processing overhead.
Multi-Level Resistive Synapses for On-Chip Neural Networks: A Physics-Based Design of a Memristive Crossbar Fabric with Quasi-Continuous Conductance States
Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars
IT (Institut Interdisciplinaire d’Innovation Technologique)(跨学科技术创新研究所);Université de Sherbrooke(舍布鲁克大学);Institute of Electronics, Microelectronics and Nanotechnology (IEMN), Université de Lille(里尔大学电子、微电子与纳米技术研究所)
LIMCA: LLM for Automating Analog In-Memory Computing Architecture Design Exploration
New Jersey Institute of Technology(新泽理工学院);Intel Corporation(英特尔公司);University of South Carolina(南卡罗来纳大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Texas at Austin(德克萨斯大学奥斯汀分校)
Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training
Department of Electrical and Computer Engineering, Cornell University, New York, NY(康奈尔大学电气与计算机工程系);IBM T. J. Watson Research Center, Yorktown Heights, NY(IBM 沃森研究中心);Rensselaer Polytechnic Institute, Troy, NY(伦塞拉尔理工学院)
Neural-PIM: Efficient Processing-In-Memory with Neural Approximation of Peripherals
Washington University in St. Louis(圣路易斯华盛顿大学);Shanghai Jiao Tong University(上海交通大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)