Deep neural network (DNN) models have become increasingly crucial components in intelligent software systems. However, training a DNN model is typically expensive in terms of both time and money. To address this issue, researchers have recently focused on reusing existing DNN models - borrowing the idea of code reuse in software engineering. However, reusing an entire model could cause extra overhead or inherits the weakness from the undesired functionalities. Hence, existing work proposes to decompose an already trained model into modules, i.e., modularizing-after-training, and enable module reuse. Since trained models are not built for modularization, modularizing-after-training incurs huge overhead and model accuracy loss. In this paper, we propose a novel approach that incorporates modularization into the model training process, i.e., modularizing-while-training (MwT). We train a model to be structurally modular through two loss functions that optimize intra-module cohesion and inter-module coupling. We have implemented the proposed approach for modularizing Convolutional Neural Network (CNN) models in this work. The evaluation results on representative models demonstrate that MwT outperforms the state-of-the-art approach. Specifically, the accuracy loss caused by MwT is only 1.13 percentage points, which is 1.76 percentage points less than that of the baseline. The kernel retention rate of the modules generated by MwT is only 14.58%, with a reduction of 74.31% over the state-of-the-art approach. Furthermore, the total time cost required for training and modularizing is only 108 minutes, half of the baseline.
NeMo: A Neuron-Level Modularizing-While-Training Approach for Decomposing DNN Models
State Key Laboratory of Complex \& Critical Software Environment, Beihang University Beijing China;Peng Cheng Laboratory Shenzhen China;National University of Singapore Singapore;Beihang University Beijing China;Hangzhou Innovation Institute of Beihang University Hangzhou China;State Key Laboratory of Complex \& Critical Software Environment, Beihang University;Peng Cheng Laboratory;National University of Singapore;Beihang University;Hangzhou Innovation Institute of Beihang University
Accelerating Multi-Model Inference by Merging DNNs of Different Weights
Seoul National University(首尔国立大学);Qualcomm AI Research(高通人工智能研究院)
已被证明在机器学习任务上表现良好的标准化DNN模型被广泛使用,并经常被原样采用以解决下游任务,形成了迁移学习范式。然而,当从GPU服务器的集群服务这种DNN模型的多个实例时,用于提高GPU利用率的现有技术(诸如GPU)是不适用的,因为模型通常由于微调而不共享权重。我们提出了一种融合多个DNN模型的技术,这些模型共享相同的架构,但具有不同的权重和不同的输入。通过将操作替换为更一般的对应操作,可以使Networds成为可能,这些操作允许一组权重仅与某组输入相关联。在ResNet-50、ResNeXt-50、BERT和XLNet上进行的实验表明,NetCore在合并32个模型实例时,可以将DNN推理时间在NVIDIA V100 GPU上加快3.6倍,在TITAN Xp GPU上加快3.0倍,同时仅使用少量额外的GPU内存。
Toward a Robust R2D2 Paradigm for Radio-interferometric Imaging: Revisiting Deep Neural Network Training and Architecture
Institute of Sensors, Signals and Systems, Heriot-Watt University, Edinburgh EH14 4AS, United Kingdom(传感器、信号与系统研究所,赫里奥特-瓦特大学);EPCC, University of Edinburgh, Potterrow, Edinburgh EH8 9BT, United Kingdom(爱丁堡大学埃平中心)
Backdoor Defense via Decoupling the Training Process
Zhejiang University(浙江大学);Tsinghua University(清华大学);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))