Deep networks consume a large amount of memory by their nature. A natural question arises can we reduce that memory requirement whilst maintaining performance. In particular, in this work we address the problem of memory efficient learning for multiple tasks. To this end, we propose a novel network architecture producing multiple networks of different configurations, termed deep virtual networks (DVNs), for different tasks. Each DVN is specialized for a single task and structured hierarchically. The hierarchical structure, which contains multiple levels of hierarchy corresponding to different numbers of parameters, enables multiple inference for different memory budgets. The building block of a deep virtual network is based on a disjoint collection of parameters of a network, which we call a unit. The lowest level of hierarchy in a deep virtual network is a unit, and higher levels of hierarchy contain lower levels' units and other additional units. Given a budget on the number of parameters, a different level of a deep virtual network can be chosen to perform the task. A unit can be shared by different DVNs, allowing multiple DVNs in a single network. In addition, shared units provide assistance to the target task with additional knowledge learned from another tasks. This cooperative configuration of DVNs makes it possible to handle different tasks in a memory-aware manner. Our experiments show that the proposed method outperforms existing approaches for multiple tasks. Notably, ours is more efficient than others as it allows memory-aware inference for all tasks.
Energy-Efficient Inference Accelerator for Memory-Augmented Neural Networks on an FPGA
Seoul National University(首尔国立大学)
记忆增强神经网络(MANN)是为问答任务而设计的。很难在为其他神经网络(NN)设计的加速器上有效地运行MANN,特别是在移动设备上,因为MANN需要循环数据路径和与外部存储器访问相关的各种类型的操作。我们实现了一个基于数据流架构的现场可编程门阵列(FPGA)的人工神经网络加速器。推理时间也减少了推理阈值,这是一个基于数据的最大内积搜索专用于自然语言的任务。对bAbI数据的测量显示,加速器的能量效率(FLOPS/kJ)比NVIDIA TITAN V GPU高出约125倍,在推理阈值化的情况下增加到140倍
Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
ParisTech Elite Institute of Technology, Shanghai Jiao Tong University(上海交通大学巴黎卓越工程师学院);School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);UM-SJTU Joint Institute, Shanghai Jiao Tong University(上海交通大学密西根学院);Shanghai Aerospace System Engineering Institute(上海航天系统工程研究所);The University of Melbourne(墨尔本大学)
Virtual iEEG from Scalp EEG: Charting the Landscape of Source Imaging, Intracranial Inference and Reconstruction
The Hong Kong Polytechnic University(香港理工大学);Chongqing University of Technology(重庆理工大学);Affiliated Lianyungang Hospital of Xuzhou Medical University(徐州医科大学附属连云港医院);Southern University of Science and Technology Hospital(南方科技大学医院)
Joint Memory Frequency and Computing Frequency Scaling for Energy-efficient DNN Inference
Beijing National Research Center for Information Science and Technology(北京信息科学国家研究中心);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系)