通过模块化与组合复用卷积神经网络模型
Reusing Convolutional Neural Network Models through Modularization and Composition
AI总结：
本文提出 CNNSplitter 与 GradSplitter，将训练好的 CNN 按类别拆分为可复用卷积核模块并通过组合修补或构建模型，实验证明其模块更小、精度损失低且复用效果良好。
摘要：
随着深度学习技术的广泛成功，许多经过训练的深度神经网络（DNN）模型如今已公开可用。然而，由于功能或性能不匹配，直接将公开 DNN 模型复用于新任务往往会失败。受软件复用中模块化与组合概念的启发，我们研究了以更细粒度方式提升 DNN 模型可复用性的可能性。具体而言，我们提出了两种名为 CNNSplitter 和 GradSplitter 的模块化方法，它们能够将一个用于 N 类分类的、已训练的卷积神经网络（CNN）模型分解为 N 个可复用的小模块。每个模块识别 N 个类别中的一个，并包含已训练 CNN 模型的一部分卷积核。随后，所得模块可被复用于修补现有 CNN 模型，或通过组合构建新的 CNN 模型。CNNSplitter 与 GradSplitter 的主要区别在于搜索方法：前者依赖遗传算法探索搜索空间，而后者采用基于梯度的搜索方法。我们在三个广泛使用的公开数据集上使用三种具有代表性的 CNN 进行实验，证明了所提方法的有效性。与 CNNSplitter 相比，GradSplitter 造成的精度损失更小，生成的模块小得多（卷积核减少 19.88%），并在修补弱模型方面取得更好结果。特别是，针对 GradSplitter 的实验表明：（1）通过修补弱模型，精确率、召回率和 F1 分数的平均提升分别为 17.13%、4.95% 和 11.47%；（2）对于新任务，与从零开始训练的模型相比，复用模块可达到相近精度（平均精度损失仅为 2.46%），且无需代价高昂的训练过程。我们的方法为 CNN 模型的快速开发与改进提供了一种可行解决方案。
英文摘要：
With the widespread success of deep learning technologies, many trained deep neural network (DNN) models are now publicly available. However, directly reusing the public DNN models for new tasks often fails due to mismatching functionality or performance. Inspired by the notion of modularization and composition in software reuse, we investigate the possibility of improving the reusability of DNN models in a more fine-grained manner. Specifically, we propose two modularization approaches named CNNSplitter and GradSplitter, which can decompose a trained convolutional neural network (CNN) model for $N$-class classification into $N$ small reusable modules. Each module recognizes one of the $N$ classes and contains a part of the convolution kernels of the trained CNN model. Then, the resulting modules can be reused to patch existing CNN models or build new CNN models through composition. The main difference between CNNSplitter and GradSplitter lies in their search methods: the former relies on a genetic algorithm to explore search space, while the latter utilizes a gradient-based search method. Our experiments with three representative CNNs on three widely-used public datasets demonstrate the effectiveness of the proposed approaches. Compared with CNNSplitter, GradSplitter incurs less accuracy loss, produces much smaller modules (19.88% fewer kernels), and achieves better results on patching weak models. In particular, experiments on GradSplitter show that (1) by patching weak models, the average improvement in terms of precision, recall, and F1-score is 17.13%, 4.95%, and 11.47%, respectively, and (2) for a new task, compared with the models trained from scratch, reusing modules achieves similar accuracy (the average loss of accuracy is only 2.46%) without a costly training process. Our approaches provide a viable solution to the rapid development and improvement of CNN models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
