如出一辙：视觉Transformer中的模块化超像素标记化
A Spitting Image: Modular Superpixel Tokenization in Vision Transformers
AI总结：
针对传统ViT网格标记化脱离图像语义的问题，提出解耦标记化与特征提取的模块化超像素标记化策略，结合内容感知标记化与不变位置嵌入，提升归因忠实度与密集预测粒度且保持分类性能，拓展了ViT的语义模型适用范围。
摘要：
传统Vision Transformer（ViT，视觉Transformer）架构采用与图像语义内容无关的基于网格的标记化方法。我们提出一种模块化超像素标记化策略，将标记化与特征提取解耦，这与当前将二者视为无差别整体的方法不同。我们采用在线内容感知标记化以及尺度和形状不变的位置嵌入开展实验与消融研究，将我们的方法与基于补丁的标记化、随机分区作为基线进行对比。结果表明，我们的方法显著提升了归因的忠实度，在零样本无监督密集预测任务上实现了像素级粒度，同时在分类任务中保持了预测性能。我们的方法提供了可与标准架构适配的模块化标记化框架，将ViT的适用范围拓展至更广泛的语义丰富模型类别。
英文摘要：
Vision Transformer (ViT) architectures traditionally employ a grid-based approach to tokenization independent of the semantic content of an image. We propose a modular superpixel tokenization strategy which decouples tokenization and feature extraction; a shift from contemporary approaches where these are treated as an undifferentiated whole. Using on-line content-aware tokenization and scale- and shape-invariant positional embeddings, we perform experiments and ablations that contrast our approach with patch-based tokenization and randomized partitions as baselines. We show that our method significantly improves the faithfulness of attributions, gives pixel-level granularity on zero-shot unsupervised dense prediction tasks, while maintaining predictive performance in classification tasks. Our approach provides a modular tokenization framework commensurable with standard architectures, extending the space of ViTs to a larger class of semantically-rich models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
