ARTA:自适应混合分辨率令牌分配用于高效密集特征提取
ARTA: Adaptive Mixed-Resolution Token Allocation for Efficient Dense Feature Extraction
AI总结:
ARTA通过自适应混合分辨率令牌分配实现高效密集特征提取,通过轻量级分配器预测需要更多精细令牌的区域,提升边界敏感度并减少计算量。
摘要:
我们提出了ARTA,一种混合分辨率粗到细的视觉Transformer,用于高效的密集特征提取。与从高分辨率(精细)令牌开始的模型不同,ARTA从低分辨率(粗略)令牌开始,并使用轻量级分配器预测需要更多精细令牌的区域。分配器迭代预测语义(类别)边界分数,并将额外令牌分配给阈值以上的补丁,使令牌密度集中在边界附近,同时保持对弱边界证据的高敏感性。这种定向分配鼓励令牌代表单一语义类别而非混合类别。混合分辨率注意力使粗粒度和细粒度令牌之间产生交互,将计算集中在语义复杂的区域,同时避免在同质区域的冗余处理。实验表明,ARTA在ADE20K和COCO-Stuff上取得最先进的结果,使用显著更少的FLOPs,并在Cityscapes上以明显更低的计算量提供具有竞争力的性能。例如,ARTA-Base在约100M参数类别中达到54.6 mIoU,同时使用更少的FLOPs和内存。
英文摘要:
We present ARTA, a mixed-resolution coarse-to-fine vision transformer for efficient dense feature extraction. Unlike models that begin with dense high-resolution (fine) tokens, ARTA starts with low-resolution (coarse) tokens and uses a lightweight allocator to predict which regions require more fine tokens. The allocator iteratively predicts a semantic (class) boundary score and allocates additional tokens to patches above a low threshold, concentrating token density near boundaries while maintaining high sensitivity to weak boundary evidence. This targeted allocation encourages tokens to represent a single semantic class rather than a mixture of classes. Mixed-resolution attention enables interaction between coarse and fine tokens, focusing computation on semantically complex areas while avoiding redundant processing in homogeneous regions. Experiments demonstrate that ARTA achieves state-of-the-art results on ADE20K and COCO-Stuff with substantially fewer FLOPs, and delivers competitive performance on Cityscapes at markedly lower compute. For example, ARTA-Base attains 54.6 mIoU on ADE20K in the ~100M-parameter class while using fewer FLOPs and less memory than comparable backbones.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。