arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.03347cs.CVcs.LGcs.NEeess.IV
收藏

SpinalNet: 具有渐进式输入的深度神经网络

SpinalNet: Deep Neural Network with Gradual Input

Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University(迪肯大学智能系统研究与创新研究所);Cairo University(开罗大学);National University of Singapore(新加坡国立大学)

H M Dipu Kabir, Moloud Abdar, Seyed Mohammad Jafar Jalali, Abbas Khosravi, Amir F Atiya, Saeid Nahavandi, Dipti Srinivasan

AI总结:

受人体躯体感觉系统启发设计的SpinalNet将网络层划分为三类分段以减少参数量,可作为DNN的全连接或分类层,在多个图像数据集上以更低计算成本取得SOTA性能。

摘要:

深度神经网络(DNNs)已在众多领域取得了当前最优性能。然而,DNNs 需要较高的计算耗时,人们始终期望以更低的计算量获得更优的性能。为此,我们研究人体躯体感觉系统,设计了一种神经网络(SpinalNet),以更少的计算量实现更高的准确率。传统神经网络的隐藏层接收前一层的输入,施加激活函数,再将结果传递到下一层。在提出的SpinalNet中,每一层被划分为三个部分:1)输入分段、2)中间分段、3)输出分段。每一层的输入分段接收一部分输入;每一层的中间分段接收前一层中间分段的输出,以及当前层输入分段的输出。其传入权重的数量显著低于传统DNNs。SpinalNet还可用作DNN的全连接层或分类层,同时支持传统学习和迁移学习。我们在大多数DNN中观察到,该网络以更低的计算成本实现了显著的误差降低。搭载SpinalNet分类层的VGG-5网络在传统学习设置下,在QMNIST、Kuzushiji-MNIST、EMNIST(Letters、Digits和Balanced)数据集上取得了当前最优(SOTA)性能;采用ImageNet预训练初始权重、搭载SpinalNet分类层的传统学习设置,在STL-10、Fruits 360、Bird225和Caltech-101数据集上取得了SOTA性能。所提出的SpinalNet的代码脚本可通过以下链接获取:https://github.com/dipuk0506/SpinalNet

英文摘要:

Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet are available at the following link: https://github.com/dipuk0506/SpinalNet

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑