Fast-ParC：为卷积神经网络与视觉Transformer捕获位置感知的全局特征
Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs
AI总结：
本文提出位置感知循环卷积ParC及其FFT加速版Fast-ParC，以即插即用方式融合ConvNets与ViTs优势，在保持低计算成本的同时扩大感受野，提升分类、检测和分割任务性能。
摘要：
近年来，Transformer模型在各个领域取得了巨大进展。在计算机视觉领域，视觉Transformer（ViTs）也成为卷积神经网络（ConvNets）的有力替代方案，但由于两者各有优点，ViTs尚未能取代ConvNets。例如，ViTs擅长利用注意力机制提取全局特征，而ConvNets由于其强大的归纳偏置，在建模局部关系方面更为高效。由此产生的一个自然想法是结合ConvNets和ViTs的优势来设计新的结构。在本文中，我们提出了一种新的基础神经网络算子，名为位置感知循环卷积（ParC）及其加速版本Fast-ParC。ParC算子通过使用全局核和循环卷积来捕获全局特征，同时通过使用位置嵌入保持位置敏感性。我们的Fast-ParC进一步利用快速傅里叶变换将ParC的O(n²)时间复杂度降低到O(n log n)。这种加速使得在具有大特征图的模型早期阶段使用全局卷积成为可能，同时仍保持与使用3x3或7x7核相当的整体计算成本。所提出的操作可以以即插即用的方式用于：1）将ViTs转换为纯ConvNet架构，以获得更广泛的硬件支持并实现更高的推理速度；2）替换ConvNets深层阶段的传统卷积，通过扩大有效感受野来提高精度。实验结果表明，我们的ParC算子可以有效扩大传统ConvNets的感受野，并且采用所提出的算子在所有三个流行的视觉任务（图像分类、目标检测和实例分割）上均有益于ViTs和ConvNet模型。
英文摘要：
Transformer models have made tremendous progress in various fields in recent years. In the field of computer vision, vision transformers (ViTs) also become strong alternatives to convolutional neural networks (ConvNets), yet they have not been able to replace ConvNets since both have their own merits. For instance, ViTs are good at extracting global features with attention mechanisms while ConvNets are more efficient in modeling local relationships due to their strong inductive bias. A natural idea that arises is to combine the strengths of both ConvNets and ViTs to design new structures. In this paper, we propose a new basic neural network operator named position-aware circular convolution (ParC) and its accelerated version Fast-ParC. The ParC operator can capture global features by using a global kernel and circular convolution while keeping location sensitiveness by employing position embeddings. Our Fast-ParC further reduces the O(n2) time complexity of ParC to O(n log n) using Fast Fourier Transform. This acceleration makes it possible to use global convolution in the early stages of models with large feature maps, yet still maintains the overall computational cost comparable with using 3x3 or 7x7 kernels. The proposed operation can be used in a plug-and-play manner to 1) convert ViTs to pure-ConvNet architecture to enjoy wider hardware support and achieve higher inference speed; 2) replacing traditional convolutions in the deep stage of ConvNets to improve accuracy by enlarging the effective receptive field. Experiment results show that our ParC op can effectively enlarge the receptive field of traditional ConvNets, and adopting the proposed op benefits both ViTs and ConvNet models on all three popular vision tasks, image classification, object
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
