arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.04466cs.CV
收藏

神经网络中的内包围

Intra-Ensemble in Neural Networks

Megvii Inc (Face++)(旷视科技(Face++));Beihang University(北京航空航天大学)

Yuan Gao, Zixiang Cai, Lei Yu

摘要:

提高模型性能一直是包括深度学习在内的机器学习中的关键问题。然而,独立的神经网络总是遭受边际效应时,堆叠更多的层。同时,集成是一个有用的技术,以进一步提高模型的性能。然而,训练多个独立的深度神经网络用于集成需要多个资源。如果是这样,是否可以只在一个神经网络中使用集成?在这项工作中,我们提出了内Ensemble,一个端到端的集成策略与随机通道重组操作训练几个子网络同时在一个神经网络。由于大多数参数是相互共享的,因此附加参数的大小是微不足道的。同时,随机信道重组显著增加了子网络的多样性,最终提高了集成性能。大量的实验和消融研究证明了内集成在各种数据集和网络结构上的适用性。

英文摘要:

Improving model performance is always the key problem in machine learning including deep learning. However, stand-alone neural networks always suffer from marginal effect when stacking more layers. At the same time, ensemble is an useful technique to further enhance model performance. Nevertheless, training several independent deep neural networks for ensemble costs multiple resources. If so, is it possible to utilize ensemble in only one neural network? In this work, we propose Intra-Ensemble, an end-to-end ensemble strategy with stochastic channel recombination operations to train several sub-networks simultaneously within one neural network. Additional parameter size is marginal since the majority of parameters are mutually shared. Meanwhile, stochastic channel recombination significantly increases the diversity of sub-networks, which finally enhances ensemble performance. Extensive experiments and ablation studies prove the applicability of intra-ensemble on various kinds of datasets and network architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑