arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.13666cs.LGcs.AR
收藏

Schrödinger's FP:深度学习训练中浮点容器的动态适配

Schrödinger's FP: Dynamic Adaptation of Floating-Point Containers for Deep Learning Training

University of Toronto(多伦多大学);Qualcomm(高通公司);QBit(1QBit公司);AMD(超威半导体公司);McMaster University(麦克马斯特大学)

Miloš Nikolić, Enrique Torres Sanchez, Jiahui Wang, Ali Hadi Zadeh, Mostafa Mahmoud, Ameer Abdelhadi, Kareem Ibrahim, Andreas Moshovos

AI总结:

针对深度学习训练中张量传输能耗高、现有窄数据表示需人工试错的问题,提出动态适配浮点格式的多类方法,结合有损压缩与可选无损压缩显著降低内存占用。

摘要:

神经网络训练过程中张量在内存间的传输占据了主要的时间与能耗。为提升能效与性能,已有研究在探索使用更窄数据表示的方法。但截至目前,这些尝试都依赖用户引导的试错才能实现收敛。我们提出的方法可让用户无需承担这一工作。我们的方法会在训练过程中动态调整用于激活值与权重的浮点容器的大小与格式,在三个维度上实现自适应:i)使用何种数据类型,ii)作用于哪个张量,以及iii)其随时间的变化方式。由于指数与尾数的含义和分布各不相同,我们针对二者分别设计了定制化方法。我们提出了两组有损方法对,可在不影响精度的前提下尽可能多地削减肥数与指数字位。Quantum Mantissa(量子尾数)与Quantum Exponent(量子指数)是两类机器学习压缩方法,它们借助梯度下降算法,以层为粒度学习最小的尾数位长与指数位长。它们能自动获知许多张量仅需1或2个尾数位、以及3或4个指数位即可。总体而言,这两种机器学习方法将内存占用降低了4.74倍。另一种方法BitWave则通过观测训练过程中损失函数的变化,在整个网络范围内调整尾数与指数位长,可实现3.19倍的内存占用降低。最后,我们提出了可选方法Gecko,它利用自然出现的不对称指数分布,对Quantum Exponent或BitWave输出的指数进行无损压缩,平均可将压缩率分别提升至5.64倍和4.56倍。

英文摘要:

The transfer of tensors from/to memory during neural network training dominates time and energy. To improve energy efficiency and performance, research has been exploring ways to use narrower data representations. So far, these attempts relied on user-directed trial-and-error to achieve convergence. We present methods that relieve users from this responsibility. Our methods dynamically adjust the size and format of the floating-point containers used for activations and weights during training, achieving adaptivity across three dimensions: i) which datatype to use, ii) on which tensor, and iii) how it changes over time. The different meanings and distributions of exponent and mantissas lead us to tailored approaches for each. We present two lossy pairs of methods to eliminate as many mantissa and exponent bits as possible without affecting accuracy. Quantum Mantissa and Quantum Exponent are machine learning compression methods that tap into the gradient descent algorithm to learn the minimal mantissa and exponent bitlengths on a per-layer granularity. They automatically learn that many tensors can use just 1 or 2 mantissa bits and 3 or 4 exponent bits. Overall, the two machine learning methods reduce the footprint by $4.74\times$. Alternatively, BitWave observes changes in the loss function during training to adjust mantissa and exponent bitlengths network-wide, yielding a $3.19\times$ reduction in footprint. Finally, we present an optional method, Gecko, to exploit the naturally emerging, lop-sided exponent distribution to losslessly compress resulting exponents from Quantum Exponent or BitWave and, on average, improve compression rates to $5.64\times$ and $4.56\times$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑