每个人都应该知道的关于视觉Transformer的三件事
Three things everyone should know about Vision Transformers
摘要：
Transformer架构在自然语言处理领域取得初步成功后，在计算机视觉领域迅速获得了广泛应用，为图像分类、检测、分割和视频分析等任务提供了最先进的结果。我们提供了三个基于简单易行的视觉Transformer变体的见解。(1)通常顺序处理的视觉Transformer的剩余层可以在某种程度上并行有效地处理，而不会明显影响精度。(2)微调注意层的权重足以使视觉Transformer适应更高的分辨率和其他分类任务。这节省了计算，降低了微调时的峰值内存消耗，并允许在任务之间共享大部分权重。(3)添加基于MLP的补丁预处理层改进了基于补丁掩蔽的Bert式自监督训练。我们使用ImageNet-1 k数据集评估了这些设计选择的影响，并在ImageNet-v2测试集上证实了我们的发现。传输性能在六个较小的数据集上进行测量。
英文摘要：
After their initial success in natural language processing, transformer architectures have rapidly gained traction in computer vision, providing state-of-the-art results for tasks such as image classification, detection, segmentation, and video analysis. We offer three insights based on simple and easy to implement variants of vision transformers. (1) The residual layers of vision transformers, which are usually processed sequentially, can to some extent be processed efficiently in parallel without noticeably affecting the accuracy. (2) Fine-tuning the weights of the attention layers is sufficient to adapt vision transformers to a higher resolution and to other classification tasks. This saves compute, reduces the peak memory consumption at fine-tuning time, and allows sharing the majority of weights across tasks. (3) Adding MLP-based patch pre-processing layers improves Bert-like self-supervised training based on patch masking. We evaluate the impact of these design choices using the ImageNet-1k dataset, and confirm our findings on the ImageNet-v2 test set. Transfer performance is measured across six smaller datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
