arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1909.13844cs.LGstat.ML
收藏

容错和硬件高效的深度神经网络的自动设计

Automated design of error-resilient and hardware-efficient deep neural networks

Bosch Corporate Research(博世企业研究院);RWTH Aachen University(亚琛工业大学);Bosch Center for Artificial Intelligence(博世人工智能中心);University of Freiburg(弗莱堡大学)

Christoph Schorn, Thomas Elsken, Sebastian Vogel, Armin Runge, Andre Guntoro, Gerd Ascheid

摘要:

将深度神经网络(DNN)应用于移动和安全关键型系统(如自动驾驶汽车),需要在硬件上实现可靠、高效的执行。正在开发优化的专用硬件加速器来实现这一目标。然而,由于现代集成电路技术的增加的复杂性及其对硬件故障(诸如随机位翻转)的敏感性,高效且可靠的硬件的设计已经变得越来越困难。因此,期望也在算法侧利用用于错误恢复和效率的优化潜力,例如,通过优化DNN的架构。由于DNN的架构有许多设计选择,对优选特性(例如低延迟下的小出错率)具有部分相反的影响,因此多目标优化策略是必要的。在本文中,我们开发了一种进化优化技术,用于自动设计硬件优化的DNN架构。为此,我们推导出一组易于计算的目标函数,使DNN架构的硬件效率和错误恢复能力的快速评估仅基于网络拓扑结构。我们观察到预测的错误恢复能力和故障注入模拟获得的实际测量之间的强相关性。此外,我们分析了两种不同的量化方案,以实现高效的DNN计算,并发现它们对错误恢复的影响存在显着差异。

英文摘要:

Applying deep neural networks (DNNs) in mobile and safety-critical systems, such as autonomous vehicles, demands a reliable and efficient execution on hardware. Optimized dedicated hardware accelerators are being developed to achieve this. However, the design of efficient and reliable hardware has become increasingly difficult, due to the increased complexity of modern integrated circuit technology and its sensitivity against hardware faults, such as random bit-flips. It is thus desirable to exploit optimization potential for error resilience and efficiency also at the algorithmic side, e.g., by optimizing the architecture of the DNN. Since there are numerous design choices for the architecture of DNNs, with partially opposing effects on the preferred characteristics (such as small error rates at low latency), multi-objective optimization strategies are necessary. In this paper, we develop an evolutionary optimization technique for the automated design of hardware-optimized DNN architectures. For this purpose, we derive a set of easily computable objective functions, which enable the fast evaluation of DNN architectures with respect to their hardware efficiency and error resilience solely based on the network topology. We observe a strong correlation between predicted error resilience and actual measurements obtained from fault injection simulations. Furthermore, we analyze two different quantization schemes for efficient DNN computation and find significant differences regarding their effect on error resilience.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑