arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.14701cs.CV
收藏

RATS!补丁通过寄存器对话:寄存器注意力Transformer中的涌现部件

RATS! Patches Talk Through Registers: Emergent Parts in Register Attention Transformers

Johns Hopkins University(约翰霍普金斯大学);Office of Naval Research, Arlington, VA(海军研究办公室,阿灵顿,弗吉尼亚州);Department of Laboratory Medicine and Pathology, Mayo Clinic, MN, USA(梅奥诊所检验医学与病理学系,明尼苏达州,美国)

Timing Yang, Predrag Neskovic, Jansen Seheult, Wenchao Han, Anand Bhattad, Alan Yuille, Feng Wang

AI总结:

提出RATS模型,通过将分类令牌分解为可学习的寄存器令牌,在L→N→N→L瓶颈中路由补丁信息,无需辅助损失或部件标注,每个寄存器自发专化为类似物体部件的原语义区域,在五个分割基准上平均mIoU提升12。

摘要:

当人类看到一只鸟时,他们识别出的远不止是“鸟”——他们看到头部、翅膀和爪子,这是一个可重复使用部件的结构化组合,这些部件可以在他们见过的每一只鸟中被识别出来。我们询问一个自监督视觉模型能否自行发现相同的组合结构。为此,我们提出了RATS(寄存器注意力Transformer),它将分类令牌分解为N个可学习的寄存器令牌,通过三步压缩-通信-广播注意力机制,在L→N→N→L瓶颈中路由补丁信息。这N个寄存器被分配到H个注意力头上,因此分配给不同头的寄存器之间不相互作用。在没有辅助损失或部件标注的情况下,每个寄存器自发地专化为一个原语义区域,其涌现结构类似于物体部件。RATS在五个分割基准上平均超过所有基线+12 mIoU,在ADE20K(+1.11 mIoU)和COCO(+0.2 AP^m)上持续提升。其寄存器字典进一步展示了跨相关类别的部件级一致性和语义接近性。我们的结果表明,RATS可能为结构化和可解释的视觉表示学习提供有用的架构先验。

英文摘要:

When humans see a bird, they recognize far more than just "bird" -- they see a head, wings, and talons, a structured assembly of reusable parts that can be identified across every bird they have ever seen. We ask whether a self-supervised visual model can discover the same compositional structure on its own. To this end, we propose RATS (Register Attention Transformers), which decomposes the classification token into N learnable register tokens that route patch information through an L->N->N->L bottleneck via a three-step compress-communicate-broadcast attention. The N registers are partitioned across the H attention heads, so that registers assigned to different heads do not interact with each other. Without auxiliary losses or part annotations, each register spontaneously specializes into a proto-semantic region whose emerging structure resembles object parts. RATS surpasses all baselines by +12 mIoU on average across five segmentation benchmarks, with consistent gains on ADE20K (+1.11 mIoU) and COCO (+0.2 AP^m). Its register dictionary further exhibits part-level consistency and semantic proximity across related categories. Our results suggest that RATS may provide a useful architectural prior for structured and interpretable visual representation learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑