arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.15425cs.CV
收藏

卷积神经网络的FocusedDropout

FocusedDropout for Convolutional Neural Network

University of Electronic Science and Technology of China(电子科技大学)

Tianshu Xie, Minghui Liu, Jiali Deng, Xuan Cheng, Xiaomin Wang, Ming Liu

摘要:

在卷积神经网络(CNN)中,dropout不能很好地工作,因为丢弃的信息在特征空间相关的卷积层中并不完全模糊。除了随机丢弃区域或通道之外,许多方法试图通过丢弃有影响的单元来克服这个缺陷。在本文中,我们提出了一种名为FocusedDropout的非随机丢弃方法,旨在使网络更专注于目标。在FocusedDropout中,我们使用一种简单但有效的方法来搜索与目标相关的特征,保留这些特征并丢弃其他特征,这与现有的方法相反。我们发现,这种新方法可以通过使网络更加专注于目标来提高网络性能。此外,在使用FocusedDropout时增加权重衰减可以避免过拟合并提高精度。实验结果表明,即使是很小的成本,10%的批次采用FocusedDropout,可以产生一个很好的性能提升超过基线的多个数据集的分类,包括CIFAR10,CIFAR100,Tiny Imagenet,并具有良好的通用性,为不同的CNN模型。

英文摘要:

In convolutional neural network (CNN), dropout cannot work well because dropped information is not entirely obscured in convolutional layers where features are correlated spatially. Except randomly discarding regions or channels, many approaches try to overcome this defect by dropping influential units. In this paper, we propose a non-random dropout method named FocusedDropout, aiming to make the network focus more on the target. In FocusedDropout, we use a simple but effective way to search for the target-related features, retain these features and discard others, which is contrary to the existing methods. We found that this novel method can improve network performance by making the network more target-focused. Besides, increasing the weight decay while using FocusedDropout can avoid the overfitting and increase accuracy. Experimental results show that even a slight cost, 10\% of batches employing FocusedDropout, can produce a nice performance boost over the baselines on multiple datasets of classification, including CIFAR10, CIFAR100, Tiny Imagenet, and has a good versatility for different CNN models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑