Computer vision methods that explicitly detect object parts and reason on them are a step towards inherently interpretable models. Existing approaches that perform part discovery driven by a fine-grained classification task make very restrictive assumptions on the geometric properties of the discovered parts; they should be small and compact. Although this prior is useful in some cases, in this paper we show that pre-trained transformer-based vision models, such as self-supervised DINOv2 ViT, enable the relaxation of these constraints. In particular, we find that a total variation (TV) prior, which allows for multiple connected components of any size, substantially outperforms previous work. We test our approach on three fine-grained classification benchmarks: CUB, PartImageNet and Oxford Flowers, and compare our results to previously published methods as well as a re-implementation of the state-of-the-art method PDiscoNet with a transformer-based backbone. We consistently obtain substantial improvements across the board, both on part discovery metrics and the downstream classification task, showing that the strong inductive biases in self-supervised ViT models require to rethink the geometric priors that can be used for unsupervised part discovery.
SIEFormer: Spectral-Interpretable and -Enhanced Transformer for Generalized Category Discovery
School of Computer Science and Engineering, Nanjing University of Science and Technology(计算机科学与工程学院,南京理工大学);School of Engineering, Newcastle University(工程学院,新castle大学);School of Computing, Newcastle University(计算学院,新castle大学)
Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut
Univ. Grenoble Alpes(格勒诺布尔阿尔卑斯大学);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔理工学院);Tencent AI Lab(腾讯人工智能实验室);LIGM(加斯帕尔·蒙日信息学实验室);Ecole des Ponts(巴黎桥路学院);Samsung AI Center, Cambridge(三星剑桥人工智能中心);MIT(麻省理工学院)
Localizing Objects with Self-Supervised Transformers and no Labels
Inria(法国国家信息与自动化研究所);DIENS(法国高等师范学院信息与计算机科学系);ENS-PSL(巴黎文理研究大学);CNRS(法国国家科学研究中心);LIGM(巴黎东大学信息与建模实验室);Ecole des Ponts(巴黎桥路学院);Univ Gustave Eiffel(Gustave 埃菲尔大学);Center for Data Science, New York University(纽约大学数据科学中心)
GraViT: Transfer Learning with Vision Transformers and MLP-Mixer for Strong Gravitational Lens Discovery
Facultad de Ingeniería, Universidad Autónoma de San Luis Potosí(圣路易斯波托西自治大学工程学院);Universidad Nacional Autónoma de México. Instituto de Astronomía(墨西哥国家自治大学天文研究所);Inter-University Centre for Astronomy and Astrophysics (IUCAA)(天文学与天体物理学跨大学研究中心);Kavli IPMU (WPI), UTIAS, The University of Tokyo(Kavli IPMU(WPI),东京大学UTIAS);Astronomy Research Group and Bosscha Observatory, FMIPA, Institut Teknologi Bandung(天文学研究组和博斯卡天文台,万隆理工学院);U-CoE AI-VLB, Institut Teknologi Bandung(人工智能与视觉学习卓越中心,万隆理工学院);University Center of Excellence for Space Science, Technology and Innovation, Institut Teknologi Bandung(空间科学、技术和创新卓越大学中心,万隆理工学院)
ViTNF: Leveraging Neural Fields to Boost Vision Transformers in Generalized Category Discovery
School of Mathematics and Information Science(数学与信息科学学院);Guangxi University(广西大学);Shanghai Institute of Applied Mathematics and Mechanics(应用数学与力学研究所);School of Mechanics and Engineering Science(力学与工程科学学院);Shanghai University(上海大学)
本文提出 ViTNF,用基于耦合静态神经场的分类器替换 ViT 的 MLP 头,并简化训练流程,在多个 GCD 基准上显著提升新类别和所有类别的准确率。