arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1911.05941cs.LGstat.ML
收藏

一种高效的面向硬件的丢弃算法

An Efficient Hardware-Oriented Dropout Algorithm

Kyushu Institute of Technology(九州工业大学)

Yoeng Jye Yeoh, Takashi Morie, Hakaru Tamukoh

摘要:

提出了一种面向硬件的丢弃算法,该算法适用于现场可编程门阵列(FPGA)实现。在深度神经网络(DNN)中,当网络过度训练并且太好地适应训练数据时,就会发生过拟合。因此,它们无法预测用作测试数据的不可见数据。丢弃是DNN中经常应用的一种常见技术,用于克服这个问题。通常,由于功率和存储器约束,在嵌入式系统中实现DNN的这种训练算法是困难的。训练DNN是功耗、时间和内存密集型的;然而,嵌入式系统需要低功耗和实时处理。FPGA具有并行处理的特点和低功耗的特点,非常适合嵌入式系统,但由于其存储空间有限和体系结构的不同,使得通用的神经网络算法难以应用。因此,我们提出了一个面向硬件的丢弃算法,可以有效地利用FPGA的特点,所需的内存较少。软件程序验证表明,该方法的性能是相同的传统的辍学,和硬件综合表明,它导致显着的资源减少。

英文摘要:

This paper proposes a hardware-oriented dropout algorithm, which is efficient for field programmable gate array (FPGA) implementation. In deep neural networks (DNNs), overfitting occurs when networks are overtrained and adapt too well to training data. Consequently, they fail in predicting unseen data used as test data. Dropout is a common technique that is often applied in DNNs to overcome this problem. In general, implementing such training algorithms of DNNs in embedded systems is difficult due to power and memory constraints. Training DNNs is power-, time-, and memory- intensive; however, embedded systems require low power consumption and real-time processing. An FPGA is suitable for embedded systems for its parallel processing characteristic and low operating power; however, due to its limited memory and different architecture, it is difficult to apply general neural network algorithms. Therefore, we propose a hardware-oriented dropout algorithm that can effectively utilize the characteristics of an FPGA with less memory required. Software program verification demonstrates that the performance of the proposed method is identical to that of conventional dropout, and hardware synthesis demonstrates that it results in significant resource reduction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑