arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.13021cs.LGstat.ML
收藏

SuperNet --一种有效的神经网络集成方法

SuperNet -- An efficient method of neural networks ensembling

AGH University of Science and Technology(AGH科技大学)

Ludwik Bukowski, Witold Dzwinel

摘要:

神经网络集成的主要缺陷是它对计算的要求非常高,特别是如果单个子模型是大型神经网络,则必须单独训练。考虑到现代DNN可以非常准确,它们已经是简单分类器的巨大集合,并且可以为任何集合构建具有类似性能的更节俭的压缩神经网络,设计昂贵的SuperNets的想法可能是值得怀疑的。人们普遍认为,集成增加了预测时间,使其没有吸引力,这可能是ML研究的主流是针对更先进和更有效的神经网络开发更好的损失函数和学习策略的原因。另一方面,所有这些因素使架构更加复杂,这可能导致过度拟合和高计算复杂性,也就是说,导致高度参数化的SuperNets集合被指责的相同缺陷。硕士论文的目标是加快合奏生成所需的执行时间。代替训练K个不准确的子模型,它们中的每一个可以表示单个DNN的训练的各个阶段(表示损失函数的各个局部最小值)[Huang等人,2017; Gripov等人,2018年]。因此,超级网络的计算性能可以与训练其单个子模型所花费的最大CPU时间相媲美,加上训练超级网络耦合因子所需的通常短得多的CPU时间。

英文摘要:

The main flaw of neural network ensembling is that it is exceptionally demanding computationally, especially, if the individual sub-models are large neural networks, which must be trained separately. Having in mind that modern DNNs can be very accurate, they are already the huge ensembles of simple classifiers, and that one can construct more thrifty compressed neural net of a similar performance for any ensemble, the idea of designing the expensive SuperNets can be questionable. The widespread belief that ensembling increases the prediction time, makes it not attractive and can be the reason that the main stream of ML research is directed towards developing better loss functions and learning strategies for more advanced and efficient neural networks. On the other hand, all these factors make the architectures more complex what may lead to overfitting and high computational complexity, that is, to the same flaws for which the highly parametrized SuperNets ensembles are blamed. The goal of the master thesis is to speed up the execution time required for ensemble generation. Instead of training K inaccurate sub-models, each of them can represent various phases of training (representing various local minima of the loss function) of a single DNN [Huang et al., 2017; Gripov et al., 2018]. Thus, the computational performance of the SuperNet can be comparable to the maximum CPU time spent on training its single sub-model, plus usually much shorter CPU time required for training the SuperNet coupling factors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑