捆绑包:使架构适应深度神经网络的改进训练
Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks
摘要:
设计神经网络架构是一项具有挑战性的任务,并且知道模型的哪些特定层必须被调整以提高性能几乎是一个谜。在本文中,我们引入了一种新的理论和度量来识别降低训练模型测试精度的层,这种识别早在训练开始时就完成了。在最坏的情况下,这样的一层可能会导致一个根本无法训练的网络。更准确地说,我们确定了那些使性能恶化的层,因为它们产生了冲突的训练包,正如我们在新颖的理论分析中所示,并辅以我们广泛的实证研究。在此基础上,提出了一种自动去除性能下降层的算法。该算法发现的架构实现了竞争力的准确性相比,对国家的最先进的架构。在保持如此高的精度的同时,我们的方法大大减少了不同计算机视觉任务的内存消耗和推理时间。
英文摘要:
Designing neural network architectures is a challenging task and knowing which specific layers of a model must be adapted to improve the performance is almost a mystery. In this paper, we introduce a novel theory and metric to identify layers that decrease the test accuracy of the trained models, this identification is done as early as at the beginning of training. In the worst-case, such a layer could lead to a network that can not be trained at all. More precisely, we identified those layers that worsen the performance because they produce conflicting training bundles as we show in our novel theoretical analysis, complemented by our extensive empirical studies. Based on these findings, a novel algorithm is introduced to remove performance decreasing layers automatically. Architectures found by this algorithm achieve a competitive accuracy when compared against the state-of-the-art architectures. While keeping such high accuracy, our approach drastically reduces memory consumption and inference time for different computer vision tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。