Subnetwork-to-go：具有动态训练和可定制推理的弹性神经网络
Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inference
AI总结：
针对多设备部署需反复训练小模型的问题，本文提出支持动态深度与宽度训练的弹性网络，可在推理时从单个大模型中按需提取任意规模子网络，并在音乐源分离任务上验证其性能与训练效率。
摘要：
将神经网络部署到不同设备或平台通常具有挑战性，尤其是在模型规模较大或模型复杂度较高时。尽管已有模型剪枝或蒸馏方法，但为了获得满足模型规模或复杂度约束的较小模型，通常仍需要执行一整轮模型训练或微调流程。受近期动态神经网络相关工作的启发，我们提出一种简单方法：训练一个大型网络，并在推理阶段根据给定的模型规模或复杂度约束灵活地从中提取子网络。我们引入了一种新方式，使大型模型能够在训练阶段以动态深度和宽度进行训练；在大型模型训练完成后，我们可以在推理阶段从中选择具有任意深度和宽度的子网络，并且与从头开始独立训练该子网络相比，其性能相对更好。在音乐源分离模型上的实验结果表明，我们提出的方法能够通过单个大型模型，在不同子网络规模和复杂度下有效提升分离性能，而且训练该大型模型所需的时间明显短于训练所有不同子网络。
英文摘要：
Deploying neural networks to different devices or platforms is in general challenging, especially when the model size is large or model complexity is high. Although there exist ways for model pruning or distillation, it is typically required to perform a full round of model training or finetuning procedure in order to obtain a smaller model that satisfies the model size or complexity constraints. Motivated by recent works on dynamic neural networks, we propose a simple way to train a large network and flexibly extract a subnetwork from it given a model size or complexity constraint during inference. We introduce a new way to allow a large model to be trained with dynamic depth and width during the training phase, and after the large model is trained we can select a subnetwork from it with arbitrary depth and width during the inference phase with a relatively better performance compared to training the subnetwork independently from scratch. Experiment results on a music source separation model show that our proposed method can effectively improve the separation performance across different subnetwork sizes and complexities with a single large model, and training the large model takes significantly shorter time than training all the different subnetworks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
