arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1802.02178cs.NE
收藏

LightNN:填补传统深度神经网络和二值化网络之间的空白

LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks

Ruizhou Ding, Zeye Liu, Rongye Shi, Diana Marculescu, R. D. Blanton

摘要:

深度神经网络(DNN)的专用集成电路(ASIC)实现由于其更高的分类速度而被许多系统采用。然而,尽管它们可能具有更好的准确性,但较大的DNN需要大量的能量和面积,从而限制了它们的广泛采用。DNN的能量消耗由存储器访问和计算驱动。二值化神经网络(BNN)作为精度和能量消耗之间的折衷,可以实现极大的能量减少,并且由于其正则化效应,对于大型DNN具有良好的精度。然而,当采用较小的DNN配置时,BNN显示出较差的准确性。在本文中,我们提出了一种新的DNN模型LightNN,它将乘法替换为一个移位或有限数量的移位和加法。对于固定的DNN配置,LightNN在稍微增加能量的情况下比BNN具有更好的准确性,但比传统DNN更节能,精度仅略低。因此,LightNN为硬件设计人员提供了更多选择,以便在精度和能耗之间进行权衡。此外,对于大型DNN配置,LightNN具有正则化效应,使其在精度上优于传统DNN。这些结论是通过实验验证使用MNIST和CIFAR-10数据集为不同的DNN配置。

英文摘要:

Application-specific integrated circuit (ASIC) implementations for Deep Neural Networks (DNNs) have been adopted in many systems because of their higher classification speed. However, although they may be characterized by better accuracy, larger DNNs require significant energy and area, thereby limiting their wide adoption. The energy consumption of DNNs is driven by both memory accesses and computation. Binarized Neural Networks (BNNs), as a trade-off between accuracy and energy consumption, can achieve great energy reduction, and have good accuracy for large DNNs due to its regularization effect. However, BNNs show poor accuracy when a smaller DNN configuration is adopted. In this paper, we propose a new DNN model, LightNN, which replaces the multiplications to one shift or a constrained number of shifts and adds. For a fixed DNN configuration, LightNNs have better accuracy at a slight energy increase than BNNs, yet are more energy efficient with only slightly less accuracy than conventional DNNs. Therefore, LightNNs provide more options for hardware designers to make trade-offs between accuracy and energy. Moreover, for large DNN configurations, LightNNs have a regularization effect, making them better in accuracy than conventional DNNs. These conclusions are verified by experiment using the MNIST and CIFAR-10 datasets for different DNN configurations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑