arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.04538cs.CVcs.AIcs.LG
收藏

PDiscoFormer:利用视觉Transformer放宽部件发现约束

PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformers

Inria(法国国家信息与自动化研究所);Univ. Montpellier(蒙彼利埃大学);Inrae(法国国家农业、食品与环境研究院);UMR TETIS(TETIS联合研究单位)

Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Diego Marcos

AI总结:

本文提出PDiscoFormer,利用自监督ViT和总变差先验放宽部件发现约束,在多个细粒度基准上显著提升部件发现与分类性能。

摘要:

显式检测对象部件并基于其进行推理的计算机视觉方法是迈向固有可解释模型的一步。现有方法通过细粒度分类任务驱动部件发现,但对所发现部件的几何属性做出了非常严格的假设;它们应当是小而紧凑的。尽管这一先验在某些情况下有用,但本文表明,预训练的基于Transformer的视觉模型(如自监督DINOv2 ViT)能够放宽这些约束。特别是,我们发现总变差(TV)先验允许任意大小的多个连通分量,其性能大幅超越先前工作。我们在三个细粒度分类基准上测试了我们的方法:CUB、PartImageNet和Oxford Flowers,并将我们的结果与先前发表的方法以及使用基于Transformer骨干的现有最优方法PDiscoNet的重新实现进行了比较。我们在部件发现指标和下游分类任务上均持续获得大幅改进,这表明自监督ViT模型中的强归纳偏置要求重新思考可用于无监督部件发现的几何先验。

英文摘要:

Computer vision methods that explicitly detect object parts and reason on them are a step towards inherently interpretable models. Existing approaches that perform part discovery driven by a fine-grained classification task make very restrictive assumptions on the geometric properties of the discovered parts; they should be small and compact. Although this prior is useful in some cases, in this paper we show that pre-trained transformer-based vision models, such as self-supervised DINOv2 ViT, enable the relaxation of these constraints. In particular, we find that a total variation (TV) prior, which allows for multiple connected components of any size, substantially outperforms previous work. We test our approach on three fine-grained classification benchmarks: CUB, PartImageNet and Oxford Flowers, and compare our results to previously published methods as well as a re-implementation of the state-of-the-art method PDiscoNet with a transformer-based backbone. We consistently obtain substantial improvements across the board, both on part discovery metrics and the downstream classification task, showing that the strong inductive biases in self-supervised ViT models require to rethink the geometric priors that can be used for unsupervised part discovery.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑