arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.10619cs.CV
收藏

具有分层池的可扩展视觉Transformer

Scalable Vision Transformers with Hierarchical Pooling

Monash University(莫纳什大学)

Zizheng Pan, Bohan Zhuang, Jing Liu, Haoyu He, Jianfei Cai

摘要:

最近提出的视觉图像Transformers(ViT)与纯注意力的图像识别任务,如图像分类取得了可喜的成绩。然而,目前的ViT模型的例行程序是在推理过程中保持一个完整长度的补丁序列,这是冗余的,缺乏层次表示。为此,我们提出了一个分层的视觉Transformer(HVT),逐步池的视觉令牌,以缩小序列长度,从而降低计算成本,类似于特征图下采样卷积神经网络(CNN)。它带来了一个很大的好处,我们可以增加模型的容量,通过缩放的深度/宽度/分辨率/补丁大小的尺寸,而不引入额外的计算复杂性,由于减少序列长度。此外,我们的经验发现,平均池视觉令牌包含更多的歧视性信息比单类令牌。为了证明我们的HVT的改进的可扩展性,我们进行了广泛的实验上的图像分类任务。在具有可比FLOP的情况下,我们的HVT在ImageNet和CIFAR-100数据集上的表现优于竞争基线。代码可在https://github.com/MonashAI/HVT上获得

英文摘要:

The recently proposed Visual image Transformers (ViT) with pure attention have achieved promising performance on image recognition tasks, such as image classification. However, the routine of the current ViT model is to maintain a full-length patch sequence during inference, which is redundant and lacks hierarchical representation. To this end, we propose a Hierarchical Visual Transformer (HVT) which progressively pools visual tokens to shrink the sequence length and hence reduces the computational cost, analogous to the feature maps downsampling in Convolutional Neural Networks (CNNs). It brings a great benefit that we can increase the model capacity by scaling dimensions of depth/width/resolution/patch size without introducing extra computational complexity due to the reduced sequence length. Moreover, we empirically find that the average pooled visual tokens contain more discriminative information than the single class token. To demonstrate the improved scalability of our HVT, we conduct extensive experiments on the image classification task. With comparable FLOPs, our HVT outperforms the competitive baselines on ImageNet and CIFAR-100 datasets. Code is available at https://github.com/MonashAI/HVT

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑