Interflow:基于注意力机制的多层特征映射聚合
Interflow: Aggregating Multi-layer Feature Mappings with Attention Mechanism
摘要:
传统上,CNN模型具有层次结构,并利用最后一层的特征映射来获得预测输出。然而,解决最佳网络深度并使中间层学习区别特征可能是困难的。针对传统的CNN模型提出了Interflow算法。Interflow根据深度将CNN分为几个阶段,并通过每个阶段的特征映射进行预测。随后,我们将这些预测分支输入到一个精心设计的注意力模块中,该模块学习这些预测分支的权重,将它们聚合并获得最终输出。Interflow对浅层和深层学习的特征进行加权融合,使各阶段的特征信息得到合理有效的处理,使中间层能够学习到更有区别的特征,增强模型的表示能力。此外,Interflow算法通过引入注意力机制,可以缓解梯度消失问题,降低网络深度选择的难度,减轻可能出现的过拟合问题。此外,它可以避免网络退化作为副产品。与原模型相比,采用Interflow的CNN模型在多个基准数据集上实现了更高的测试精度。
英文摘要:
Traditionally, CNN models possess hierarchical structures and utilize the feature mapping of the last layer to obtain the prediction output. However, it can be difficulty to settle the optimal network depth and make the middle layers learn distinguished features. This paper proposes the Interflow algorithm specially for traditional CNN models. Interflow divides CNNs into several stages according to the depth and makes predictions by the feature mappings in each stage. Subsequently, we input these prediction branches into a well-designed attention module, which learns the weights of these prediction branches, aggregates them and obtains the final output. Interflow weights and fuses the features learned in both shallower and deeper layers, making the feature information at each stage processed reasonably and effectively, enabling the middle layers to learn more distinguished features, and enhancing the model representation ability. In addition, Interflow can alleviate gradient vanishing problem, lower the difficulty of network depth selection, and lighten possible over-fitting problem by introducing attention mechanism. Besides, it can avoid network degradation as a byproduct. Compared with the original model, the CNN model with Interflow achieves higher test accuracy on multiple benchmark datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。