We introduce Dynamic Deep Neural Networks (D2NN), a new type of feed-forward deep neural network that allows selective execution. Given an input, only a subset of D2NN neurons are executed, and the particular subset is determined by the D2NN itself. By pruning unnecessary computation depending on input, D2NNs provide a way to improve computational efficiency. To achieve dynamic selective execution, a D2NN augments a feed-forward deep neural network (directed acyclic graph of differentiable modules) with controller modules. Each controller module is a sub-network whose output is a decision that controls whether other modules can execute. A D2NN is trained end to end. Both regular and controller modules in a D2NN are learnable and are jointly trained to optimize both accuracy and efficiency. Such training is achieved by integrating backpropagation with reinforcement learning. With extensive experiments of various D2NN architectures on image classification tasks, we demonstrate that D2NNs are general and flexible, and can effectively optimize accuracy-efficiency trade-offs.
Selective Ensemble Based on Preference-Directed Multi-Objective Bandits
Department of Computer Science, Zhejiang Gongshang University(浙江工商大学计算机科学与技术学院);Center for Advanced Intelligence Project, RIKEN(日本理化学研究所先进智能研究中心);Graduate School of Frontier Sciences, The University of Tokyo(东京大学新领域创成科学研究科);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院)
University of Toronto & Vector Institute(多伦多大学及向量研究所);Massachusetts Institute of Technology(麻省理工学院);CISPA Helmholtz Center for Information Security(CISPA海德堡信息安全中心);Ericsson(爱立信)
Measuring what Really Matters: Optimizing Neural Networks for TinyML
ETH Zürich(苏黎世联邦理工学院);RWTH Aachen(亚琛工业大学)
随着廉价的计算和存储资源的激增,神经网络(NN)在架构和计算复杂性方面经历了前所未有的增长。将NN引入资源受限的设备可以实现经济高效的部署、广泛的可用性和敏感数据的保存。这项工作解决了将机器学习引入MCU的挑战,我们专注于无处不在的ARM Cortex-M架构。优化方法、软件框架和MCU硬件架构对推理延迟和能耗等关键性能指标的详细影响和权衡,以前尚未针对TensorFlow Lite Micro等最先进的框架进行深入研究。我们发现,经验调查,衡量可感知的指标-性能体验的用户-是必不可少的,因为专门的指令和层类型的影响可能是微妙的。为此,我们提出了一个实现感知的设计作为一个具有成本效益的方法进行验证和基准测试。利用我们开发的工具链,我们演示了如何通过系统地优化NN以适应其目标应用场景来改进资源受限设备上的现有NN部署。