In this work, we propose "TimeFloats," an efficient train-in-memory architecture that performs 8-bit floating-point scalar product operations in the time domain. While building on the compute-in-memory paradigm's integrated storage and inferential computations, TimeFloats additionally enables floating-point computations, thus facilitating DNN training within the same memory structures. Traditional compute-in-memory approaches with conventional ADCs and DACs face challenges such as higher power consumption and increased design complexity, especially at advanced CMOS nodes. In contrast, TimeFloats leverages time-domain signal processing to avoid conventional domain converters. It operates predominantly with digital building blocks, reducing power consumption and noise sensitivity while enabling high-resolution computations and easier integration with conventional digital circuits. Our simulation results demonstrate an energy efficiency of 22.1 TOPS/W while evaluating the design on 15 nm CMOS technology.
FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding
Harvard University（哈佛大学）；Franklin and Marshall College（富兰克林与马歇尔学院）
块浮点（BFP）可以通过经由跨一组值的共享指数提供宽动态范围来有效地支持深度神经网络（DNN）训练的量化。在本文中，我们提出了一种用于DNN的快速第一，准确第二训练（FAST）系统，其中权重，激活和梯度以BFP表示。FAST支持使用可变精度BFP输入操作数的矩阵乘法，从而在整个训练过程中实现DNN精度的增量增加。通过提高训练迭代和DNN层的BFP精度，FAST可以大大缩短训练时间，同时减少整体硬件资源使用。我们的FAST乘法器-累加器（fMAC）支持多个BFP精度下的点积计算。我们在具有不同数据集的多个DNN上验证了我们的FAST系统，在基于\textbf{mixed-precision or block}浮点数系统的先前工作中，在单芯片平台上的训练中展示了2-6个$\times$加速，同时在验证准确性方面实现了类似的性能。
Trinity College Dublin, The University of Dublin（都柏林大学圣三一学院）；The School of Computer Science and Statistics（计算机科学与统计学院）；Institute of Technology Carlow（卡洛理工学院）
Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；AMD Research and Advanced Development（AMD研究与高级开发部）；Tampere University（坦佩雷大学）；AMD（AMD公司）
