arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.11373cs.LGcs.CV
收藏

用于FC层的ProgressiveSpinalNet架构

ProgressiveSpinalNet architecture for FC layers

IIT Jodhpur(印度理工学院焦特布尔分校)

Praveen Chopra

摘要:

在深度学习模型中,FC(全连接)层对于基于从先前层学习的特征的输入分类具有最重要的作用。FC层具有最高数量的参数,并且微调这些大量的参数会消耗大部分计算资源,因此本文旨在显着减少这些大量的参数,同时提高性能。其动机受到SpinalNet和其他生物架构的启发。所提出的架构在输入到输出层之间有一个梯度高速公路,这解决了深度网络中梯度递减的问题。在这种情况下,所有层都接收来自先前层的输入以及CNN层输出,并且这样所有层都有助于最后一层的决策。这种方法提高了SpinalNet架构的分类性能,并在许多数据集上具有SOTA性能,如Caltech101,KMNIST,QMNIST和EMNIST。源代码可在https://github.com/praveenchopra/ProgressiveSpinalNet上获得。

英文摘要:

In deeplearning models the FC (fully connected) layer has biggest important role for classification of the input based on the learned features from previous layers. The FC layers has highest numbers of parameters and fine-tuning these large numbers of parameters, consumes most of the computational resources, so in this paper it is aimed to reduce these large numbers of parameters significantly with improved performance. The motivation is inspired from SpinalNet and other biological architecture. The proposed architecture has a gradient highway between input to output layers and this solves the problem of diminishing gradient in deep networks. In this all the layers receives the input from previous layers as well as the CNN layer output and this way all layers contribute in decision making with last layer. This approach has improved classification performance over the SpinalNet architecture and has SOTA performance on many datasets such as Caltech101, KMNIST, QMNIST and EMNIST. The source code is available at https://github.com/praveenchopra/ProgressiveSpinalNet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑