arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.24159cs.CVcs.LG
收藏

RAViT: 分辨率自适应视觉变换器

RAViT: Resolution-Adaptive Vision Transformer

INSA Lyon, CNRS, Université Claude Bernard Lyon 1, LIRIS, UMR5205(里尔森 Lyon,法国国家科学研究中心,克劳德·贝尔纳 Lyon 大学,LIRIS,UMR5205)

Martial Guidez, Stefan Duffner, Christophe Garcia

AI总结:

RAViT通过多分支网络和早期退出机制,在降低计算成本的同时保持高精度,实现图像分类任务的高效处理。

摘要:

视觉变换器最近在计算机视觉领域取得了突破,展示了在众多应用中精确度方面的出色表现。然而,与卷积神经网络等替代方法相比,其计算成本非常高。为了解决这个问题,我们提出了一种名为RAViT的新框架,该框架基于多分支网络,该网络在不同分辨率的多个相同图像副本上进行操作,以在保持整体准确性的同时降低计算成本。此外,我们的框架包括一个早期退出机制,使模型具有适应性,并允许在运行时选择适当的准确性和计算成本之间的权衡。例如,在双分支架构中,原始图像首先被调整大小以降低其分辨率,然后使用第一个变换器对其进行预测,所得预测结果与原始大小图像一起用于在第二个变换器上进行最终预测,其计算量比传统视觉变换器架构更少。早期退出过程使模型能够在中间分支上做出最终预测,从而节省更多计算。我们在CIFAR-10、Tiny ImageNet和ImageNet上评估了我们的方法。我们获得了与传统视觉变换器模型相当的准确度,仅使用约70%的FLOPs。

英文摘要:

Vision transformers have recently made a breakthrough in computer vision showing excellent performance in terms of precision for numerous applications. However, their computational cost is very high compared to alternative approaches such as Convolutional Neural Networks. To address this problem, we propose a novel framework for image classification called RAViT based on a multi-branch network that operates on several copies of the same image with different resolutions to reduce the computational cost while preserving the overall accuracy. Furthermore, our framework includes an early exit mechanism that makes our model adaptive and allows to choose the appropriate trade-off between accuracy and computational cost at run-time. For example in a two-branch architecture, the original image is first resized to reduce its resolution, then a prediction is performed on it using a first transformer and the resulting prediction is reused together with the original-size image to perform a final prediction on a second transformer with less computation than a classical Vision transformer architecture. The early-exit process allows the model to make a final prediction at intermediate branches, saving even more computation. We evaluated our approach on CIFAR-10, Tiny ImageNet, and ImageNet. We obtained an equivalent accuracy to the classical Vision transformer model with only around 70% of FLOPs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑