arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.13242cs.LGstat.ML
收藏

WrapNet:基于超低分辨率算法的神经网络推理

WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic

University of Maryland(马里兰大学);Cornell University(康奈尔大学)

Renkun Ni, Hong-min Chu, Oscar Castañeda, Ping-yeh Chiang, Christoph Studer, Tom Goldstein

摘要:

低分辨率神经网络用很少的比特来表示权重和激活,大大降低了乘法的复杂性。尽管如此,这些乘积是使用高分辨率(通常为32位)加法来累加的,当使用极端量化(例如,二进制权重)。为了进一步优化推理,我们提出了一种方法,使神经网络适应使用低分辨率(8位)的加法在递归中,实现分类精度与他们的32位同行。我们通过插入一个循环激活层以及一个溢出惩罚正则化器来实现对低分辨率积累的恢复。我们证明了我们的方法在软件和硬件平台上的有效性。

英文摘要:

Low-resolution neural networks represent both weights and activations with few bits, drastically reducing the multiplication complexity. Nonetheless, these products are accumulated using high-resolution (typically 32-bit) additions, an operation that dominates the arithmetic complexity of inference when using extreme quantization (e.g., binary weights). To further optimize inference, we propose a method that adapts neural networks to use low-resolution (8-bit) additions in the accumulators, achieving classification accuracy comparable to their 32-bit counterparts. We achieve resilience to low-resolution accumulation by inserting a cyclic activation layer, as well as an overflow penalty regularizer. We demonstrate the efficacy of our approach on both software and hardware platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑