arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1809.05606cs.LGstat.ML
收藏

提高DCNN性能的稠密层非迭代重计算

Non-iterative recomputation of dense layers for performance improvement of DCNN

University of Windsor(温莎大学);Lakehead University(湖首大学)

Yimin Yang, Q. M. Jonathan Wu, Xiexing Feng, Thangarajah Akilan

摘要:

迭代学习方法已成为训练深度卷积神经网络(DCNN)的范例。然而,利用非迭代学习策略可以加速DCNN的训练过程,并且令人惊讶的是,这种方法很少被深度学习(DL)社区探索。这促使本文引入一种非迭代学习策略,该策略消除了DCNN顶部密集或全连接(FC)层的反向传播(BP),从而降低了训练时间并提高了性能。该方法利用Moore-Penrose逆将当前残差拉回到每个FC层,生成良好的广义特征。然后使用重新计算的特征,即,新的广义特征是根据Moore-Penrose逆计算FC层的权值。我们在六个广泛接受的对象识别基准数据集上评估了所提出的方法:Scene-15,CIFAR-10,CIFAR-100,SUN-397,Places 365和ImageNet。实验结果表明,该方法获得了显着的改善,超过30个国家的最先进的方法。有趣的是,它还表明,使用所提出的方法的任何DCNN都可以提供比基于BP的原始训练的相同网络更好的性能。

英文摘要:

An iterative method of learning has become a paradigm for training deep convolutional neural networks (DCNN). However, utilizing a non-iterative learning strategy can accelerate the training process of the DCNN and surprisingly such approach has been rarely explored by the deep learning (DL) community. It motivates this paper to introduce a non-iterative learning strategy that eliminates the backpropagation (BP) at the top dense or fully connected (FC) layers of DCNN, resulting in, lower training time and higher performance. The proposed method exploits the Moore-Penrose Inverse to pull back the current residual error to each FC layer, generating well-generalized features. Then using the recomputed features, i.e., the new generalized features the weights of each FC layer is computed according to the Moore-Penrose Inverse. We evaluate the proposed approach on six widely accepted object recognition benchmark datasets: Scene-15, CIFAR-10, CIFAR-100, SUN-397, Places365, and ImageNet. The experimental results show that the proposed method obtains significant improvements over 30 state-of-the-art methods. Interestingly, it also indicates that any DCNN with the proposed method can provide better performance than the same network with its original training based on BP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑