MemNet:记忆效率引导的增强修剪学习神经结构搜索
MemNet: Memory-Efficiency Guided Neural Architecture Search with Augment-Trim learning
摘要:
最近关于自动神经结构搜索的研究已经证明了显着的性能,与手工制作的神经结构竞争甚至更好。然而,大多数现有的网络架构倾向于使用残余,并行结构和连接块之间的浅和深的功能来构建一个大的网络。这需要大量的内存来存储权重和特征图。这对于移动设备和嵌入式设备来说是具有挑战性的,因为它们可能没有足够的内存来使用所设计的大型网络模型执行推理。为了缩小这一差距,我们提出了MemNet,一个基于增强修剪学习的神经网络搜索框架,不仅优化了性能,而且还优化了内存需求。具体而言,它采用基于内存消耗的排名分数,强制内存消耗的上限,用于导航搜索过程。实验结果表明,与现有的高效设计方法相比,MemNet可以找到一种具有竞争力的准确率和平均节省24.17%内存的体系结构。
英文摘要:
Recent studies on automatic neural architectures search have demonstrated significant performance, competitive to or even better than hand-crafted neural architectures. However, most of the existing network architecture tend to use residual, parallel structures and concatenation block between shallow and deep features to construct a large network. This requires large amounts of memory for storing both weights and feature maps. This is challenging for mobile and embedded devices since they may not have enough memory to perform inference with the designed large network model. To close this gap, we propose MemNet, an augment-trim learning-based neural network search framework that optimizes not only performance but also memory requirement. Specifically, it employs memory consumption based ranking score which forces an upper bound on memory consumption for navigating the search process. Experiment results show that, as compared to the state-of-the-art efficient designing methods, MemNet can find an architecture which can achieve competitive accuracy and save an average of 24.17% on the total memory needed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。