arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.04976cs.CVcs.AI
收藏

双视觉Transformer

Dual Vision Transformer

JD Explore Academy(京东探索研究院);Ryerson University(瑞尔森大学)

Ting Yao, Yehao Li, Yingwei Pan, Yu Wang, Xiao-Ping Zhang, Tao Mei

摘要:

先前的工作已经提出了几种策略来减少自我注意力机制的计算成本。这些工作中的许多考虑将自注意过程分解成区域和局部特征提取过程,每个过程引起小得多的计算复杂性。然而,区域信息通常仅以由于下采样而丢失的不期望的信息为代价来实现。在本文中,我们提出了一种新的Transformer架构,旨在减轻成本问题,命名为双视觉Transformer(Dual-ViT)。新的体系结构采用了一个关键的语义路径,可以更有效地压缩令牌向量到全局语义,降低了复杂性的顺序。这样的压缩的全局语义,然后作为有用的先验信息,在学习更精细的像素级细节,通过另一个构建的像素路径。然后将语义路径和像素路径整合在一起并进行联合训练,通过两个路径并行传播增强的自我注意信息。Dual-ViT因此能够在不损害太多精度的情况下降低计算复杂性。我们的经验表明,双ViT提供了优越的准确性比SOTA Transformer架构,降低了训练的复杂性。源代码可从\url{https://github.com/YehLi/ImageNetModel}获得。

英文摘要:

Prior works have proposed several strategies to reduce the computational cost of self-attention mechanism. Many of these works consider decomposing the self-attention procedure into regional and local feature extraction procedures that each incurs a much smaller computational complexity. However, regional information is typically only achieved at the expense of undesirable information lost owing to down-sampling. In this paper, we propose a novel Transformer architecture that aims to mitigate the cost issue, named Dual Vision Transformer (Dual-ViT). The new architecture incorporates a critical semantic pathway that can more efficiently compress token vectors into global semantics with reduced order of complexity. Such compressed global semantics then serve as useful prior information in learning finer pixel level details, through another constructed pixel pathway. The semantic pathway and pixel pathway are then integrated together and are jointly trained, spreading the enhanced self-attention information in parallel through both of the pathways. Dual-ViT is henceforth able to reduce the computational complexity without compromising much accuracy. We empirically demonstrate that Dual-ViT provides superior accuracy than SOTA Transformer architectures with reduced training complexity. Source code is available at \url{https://github.com/YehLi/ImageNetModel}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑