使用Mesh自适应直接搜索算法扩展深度网络
Scaling Deep Networks with the Mesh Adaptive Direct Search algorithm
摘要:
深度神经网络越来越大。它们在边缘和物联网设备上的实现变得更具挑战性,并促使社区设计具有类似性能的更轻版本。诸如\emph{reinforcement learning}和\emph{evolutionary computing}的标准自动设计工具基本上依赖于目标函数的廉价评估。在神经网络的设计背景下,这个目标是训练后的准确性,这是昂贵的和耗时的评估。我们使用\emph{Mesh Adaptive Direct Search}(MADS)算法自动设计用于图像分类的轻型深度神经网络,这是一种成熟的无导数优化方法,即使在存在约束的情况下,也可以有效地解释目标函数探索设计空间的昂贵黑箱性质。
英文摘要:
Deep neural networks are getting larger. Their implementation on edge and IoT devices becomes more challenging and moved the community to design lighter versions with similar performance. Standard automatic design tools such as \emph{reinforcement learning} and \emph{evolutionary computing} fundamentally rely on cheap evaluations of an objective function. In the neural network design context, this objective is the accuracy after training, which is expensive and time-consuming to evaluate. We automate the design of a light deep neural network for image classification using the \emph{Mesh Adaptive Direct Search}(MADS) algorithm, a mature derivative-free optimization method that effectively accounts for the expensive blackbox nature of the objective function to explore the design space, even in the presence of constraints.Our tests show competitive compression rates with reduced numbers of trials.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。