arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.02301cs.CVcs.LG
收藏

用于低成本自集成的网络裂变集成

Network Fission Ensembles for Low-Cost Self-Ensembles

School of Integrated Technology, Yonsei University(延世大学融合技术学院)

Hojung Lee, Jong-Seok Lee

AI总结:

提出 Network Fission Ensembles (NFE) 方法,通过将传统网络转换为多出口结构实现低成本集成学习与推理,无需额外网络即可提升性能。

摘要:

近期用于图像分类的集成学习方法已被证明能以较低的额外成本提高分类准确率。然而,它们在集成推理时仍需要多个训练好的模型,当模型规模增大时,这最终会成为显著的负担。在本文中,我们提出了一种低成本的集成学习与推理方法,称为 Network Fission Ensembles (NFE),通过将传统网络本身转换为多出口结构来实现。从给定的初始网络开始,我们首先剪枝部分权重以减轻训练负担。然后,我们将剩余权重分组为若干集合,并使用每个集合创建多个辅助路径以构建多出口。我们将此过程称为 Network Fission。通过这种方式,可以从单个网络中获得多个输出,从而实现集成学习。由于此过程仅将现有网络结构更改为多出口而未使用额外网络,因此集成学习和推理没有额外的计算负担。此外,通过从所有出口的多个损失中进行学习,多出口通过正则化提升了性能,并且即使网络稀疏度增加也能实现高性能。凭借我们简单而有效的方法,与现有集成方法相比,我们取得了显著提升。代码可在 https://github.com/hjdw2/NFE 获取。

英文摘要:

Recent ensemble learning methods for image classification have been shown to improve classification accuracy with low extra cost. However, they still require multiple trained models for ensemble inference, which eventually becomes a significant burden when the model size increases. In this paper, we propose a low-cost ensemble learning and inference, called Network Fission Ensembles (NFE), by converting a conventional network itself into a multi-exit structure. Starting from a given initial network, we first prune some of the weights to reduce the training burden. We then group the remaining weights into several sets and create multiple auxiliary paths using each set to construct multi-exits. We call this process Network Fission. Through this, multiple outputs can be obtained from a single network, which enables ensemble learning. Since this process simply changes the existing network structure to multi-exits without using additional networks, there is no extra computational burden for ensemble learning and inference. Moreover, by learning from multiple losses of all exits, the multi-exits improve performance via regularization, and high performance can be achieved even with increased network sparsity. With our simple yet effective method, we achieve significant improvement compared to existing ensemble methods. The code is available at https://github.com/hjdw2/NFE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑