arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.13181cs.ARcs.AI
收藏

对数域学习:基于随机梯度下降的亚阈值模拟AI加速器

Learning in Log-Domain: Subthreshold Analog AI Accelerator Based on Stochastic Gradient Descent

Imperial College London(伦敦帝国学院)

Momen K Tageldeen, Yacine Belgaid, Vivek Mohan, Zhou Wang, Emmanuel M Drakakis

AI总结:

本文提出一种基于对数域亚阈值电路和易失性存储器的模拟加速器,用于带L2正则化的随机梯度下降训练,显著降低面积和功耗,误差低于0.87%,支持8位精度。

摘要:

AI模型的快速普及,加上对边缘部署日益增长的需求,使得开发既高性能又节能的AI硬件成为必要。本文提出了一种新颖的模拟加速器架构,专为使用带L2正则化的随机梯度下降(SGDr)的AI/ML训练工作负载而设计。该架构利用亚阈值MOS中的对数域电路,并集成了易失性存储器。我们建立了一个在连续时域中求解SGDr的数学框架,并详细描述了将SGDr学习方程映射到对数域电路的过程。通过在模拟域中运行并利用弱反型,与数字实现相比,所提出的设计在晶体管面积和功耗方面实现了显著降低。实验结果表明,该架构紧密逼近理想行为,均方误差低于0.87%,精度低至8位。此外,该架构支持广泛的超参数。这项工作为具有片上训练能力的节能模拟AI硬件铺平了道路。

英文摘要:

The rapid proliferation of AI models, coupled with growing demand for edge deployment, necessitates the development of AI hardware that is both high-performance and energy-efficient. In this paper, we propose a novel analog accelerator architecture designed for AI/ML training workloads using stochastic gradient descent with L2 regularization (SGDr). The architecture leverages log-domain circuits in subthreshold MOS and incorporates volatile memory. We establish a mathematical framework for solving SGDr in the continuous time domain and detail the mapping of SGDr learning equations to log-domain circuits. By operating in the analog domain and utilizing weak inversion, the proposed design achieves significant reductions in transistor area and power consumption compared to digital implementations. Experimental results demonstrate that the architecture closely approximates ideal behavior, with a mean square error below 0.87% and precision as low as 8 bits. Furthermore, the architecture supports a wide range of hyperparameters. This work paves the way for energy-efficient analog AI hardware with on-chip training capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑