arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.00245cs.SEcs.AI
收藏

通过模型重构实现深度神经网络模型复用

Reusing Deep Neural Network Models through Model Re-engineering

Beihang University(北京航空航天大学);Chongqing University(重庆大学)

Binhang Qi, Hailong Sun, Xiang Gao, Hongyu Zhang, Zhaotian Li, Xudong Liu

AI总结:

针对DNN模型复用中存在的开销过大和继承缺陷问题,提出模型重构工具SeaM,利用基于梯度的搜索方法仅保留与目标问题相关的权重,在大幅降低推理时间和缺陷继承的同时提升了分类准确率。

摘要:

训练深度神经网络(DNN)模型已成为当今软件开发中的一项重要任务,但在计算资源和时间方面往往成本高昂。受软件复用的启发,通过复用现有模型来构建DNN模型近期受到越来越多的关注。现有的DNN模型复用方法存在两个主要局限性:1)复用整个模型,而实际上仅需模型功能(标签)的一小部分,会导致大量开销(例如推理的计算和时间成本);2)模型复用会继承被复用模型的缺陷和弱点,从而使新系统面临安全攻击的威胁。为了解决上述问题,我们提出了SeaM,这是一种通过重构已训练的DNN模型来提升其可复用性的工具。具体而言,给定一个目标问题和一个已训练的模型,SeaM利用基于梯度的搜索方法来搜索与目标问题相关的模型权重。随后,仅保留相关权重的重构模型被复用以解决目标问题。在广泛使用的模型上的评估结果表明,由SeaM产生的重构模型仅包含原始模型10.11%的权重,使得推理时间减少了42.41%。对于目标问题,重构模型在分类准确率上甚至比原始模型高出5.85%。此外,与复用整个模型相比,复用重构模型平均减少了57%的缺陷继承。我们相信,我们提出的减少复用开销和缺陷继承的方法,是迈向实用模型复用的重要一步。

英文摘要:

Training deep neural network (DNN) models, which has become an important task in today's software development, is often costly in terms of computational resources and time. With the inspiration of software reuse, building DNN models through reusing existing ones has gained increasing attention recently. Prior approaches to DNN model reuse have two main limitations: 1) reusing the entire model, while only a small part of the model's functionalities (labels) are required, would cause much overhead (e.g., computational and time costs for inference), and 2) model reuse would inherit the defects and weaknesses of the reused model, and hence put the new system under threats of security attack. To solve the above problem, we propose SeaM, a tool that re-engineers a trained DNN model to improve its reusability. Specifically, given a target problem and a trained model, SeaM utilizes a gradient-based search method to search for the model's weights that are relevant to the target problem. The re-engineered model that only retains the relevant weights is then reused to solve the target problem. Evaluation results on widely-used models show that the re-engineered models produced by SeaM only contain 10.11% weights of the original models, resulting 42.41% reduction in terms of inference time. For the target problem, the re-engineered models even outperform the original models in classification accuracy by 5.85%. Moreover, reusing the re-engineered models inherits an average of 57% fewer defects than reusing the entire model. We believe our approach to reducing reuse overhead and defect inheritance is one important step forward for practical model reuse.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑