Analog electronic and optical computing exhibit tremendous advantages over digital computing for accelerating deep learning when operations are executed at low precision. In this work, we derive a relationship between analog precision, which is limited by noise, and digital bit precision. We propose extending analog computing architectures to support varying levels of precision by repeating operations and averaging the result, decreasing the impact of noise. Such architectures enable programmable tradeoffs between precision and other desirable performance metrics such as energy efficiency or throughput. To utilize dynamic precision, we propose a method for learning the precision of each layer of a pre-trained model without retraining network weights. We evaluate this method on analog architectures subject to a variety of noise sources such as shot noise, thermal noise, and weight noise and find that employing dynamic precision reduces energy consumption by up to 89% for computer vision models such as Resnet50 and by 24% for natural language processing models such as BERT. In one example, we apply dynamic precision to a shot-noise limited homodyne optical neural network and simulate inference at an optical energy consumption of 2.7 aJ/MAC for Resnet50 and 1.6 aJ/MAC for BERT with <2% accuracy degradation.
An Analog Neural Network Computing Engine using CMOS-Compatible Charge-Trap-Transistor (CTT)
University of California, Los Angeles（加州大学洛杉矶分校）；University of California, Irvine（加州大学欧文分校）；National Chiao Tung University（国立交通大学）；Kneron, Inc（Kneron公司）
Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Fudan University（复旦大学）；Defense Innovation Institute, Academy of Military Sciences（军事科学院国防创新研究院）
KALAM: toolKit for Automating high-Level synthesis of Analog computing systeMs
Indian Institute of Science（印度科学学院）；Sardar Vallabbhai National Institute of Technology（萨达尔·瓦拉巴伊国家理工学院）；Manipal Institute of Technology（马尼帕尔理工学院）；Washington University in St. Louis（圣路易斯华盛顿大学）
AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing
Univ. Polytechnique Hauts-de-France（上法兰西理工大学）；IBM Research Europe（IBM欧洲研究院）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；IBM Research Almaden（IBM阿尔马登研究中心）
