深度神经网络架构的数制系统:综述
Number Systems for Deep Neural Network Architectures: A Survey
AI总结:
本文综述了用于深度神经网络数据表示的各种传统与非传统数制系统,分析其对性能和硬件设计的影响、相关挑战及解决方案,并展望未来研究趋势。
摘要:
深度神经网络(DNNs)已成为众多人工智能应用的关键组成部分。在自动驾驶、健康应用等场景中,DNNs有时表现出甚至优于人类的性能。由于其计算复杂度高,在资源受限设备上部署DNNs仍面临与计算复杂度、能效、延迟和成本相关的诸多挑战。为此,学术界和工业界正在推进多个研究方向,以加速并高效实现DNNs。其中一个重要方向是确定DNN处理中涉及的海量数据的适当数据表示。研究发现,使用传统数制系统对DNNs而言并非最优。相反,大量研究聚焦于探索合适的数制系统。本文旨在对用于更高效表示DNN数据的替代数制系统进行全面综述和讨论。文中讨论了用于DNNs的各种数制系统(传统/非传统),并考虑了这些数制系统对DNNs性能和硬件设计的影响。此外,本文重点阐述了每种数制系统所面临的挑战以及为解决这些挑战而提出的各种方案。读者将能够理解高效数制系统对DNN的重要性,了解广泛用于DNN的数制系统,理解各种数制系统之间的权衡,并考虑影响数制系统对DNN性能影响的各种设计因素。此外,本文还将重点介绍近期趋势和相关研究机会。
英文摘要:
Deep neural networks (DNNs) have become an enabling component for a myriad of artificial intelligence applications. DNNs have shown sometimes superior performance, even compared to humans, in cases such as self-driving, health applications, etc. Because of their computational complexity, deploying DNNs in resource-constrained devices still faces many challenges related to computing complexity, energy efficiency, latency, and cost. To this end, several research directions are being pursued by both academia and industry to accelerate and efficiently implement DNNs. One important direction is determining the appropriate data representation for the massive amount of data involved in DNN processing. Using conventional number systems has been found to be sub-optimal for DNNs. Alternatively, a great body of research focuses on exploring suitable number systems. This article aims to provide a comprehensive survey and discussion about alternative number systems for more efficient representations of DNN data. Various number systems (conventional/unconventional) exploited for DNNs are discussed. The impact of these number systems on the performance and hardware design of DNNs is considered. In addition, this paper highlights the challenges associated with each number system and various solutions that are proposed for addressing them. The reader will be able to understand the importance of an efficient number system for DNN, learn about the widely used number systems for DNN, understand the trade-offs between various number systems, and consider various design aspects that affect the impact of number systems on DNN performance. In addition, the recent trends and related research opportunities will be highlighted
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。