蒸馏自监督视觉Transformer用于弱监督少样本分类与分割
Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation
AI总结：
本文利用自监督预训练视觉Transformer的token相关性，通过自注意力和伪标签增强器实现弱监督少样本分类与分割，在Pascal-5i和COCO-20i上显著提升性能。
摘要：
我们通过利用以自监督方式预训练的视觉Transformer（ViT）来解决弱监督少样本图像分类与分割任务。我们提出的方法从自监督ViT中提取token表示，并通过自注意力利用它们之间的相关性，经由分离的任务头生成分类和分割预测。我们的模型能够在训练期间仅使用图像级标签、缺少像素级标签的情况下，有效地学习执行分类和分割。为此，它使用由自监督ViT骨干生成的token所构建的注意力图作为像素级伪标签。我们还探索了一种具有“混合”监督的实际设置，其中少量训练图像包含真实的像素级标签，其余图像仅具有图像级标签。针对这种混合设置，我们提出使用一个伪标签增强器来改进伪标签，该增强器利用可用的真实像素级标签进行训练。在Pascal-5i和COCO-20i上的实验表明，在各种监督设置下，尤其是在几乎没有像素级标签可用时，性能均有显著提升。
英文摘要：
We address the task of weakly-supervised few-shot image classification and segmentation, by leveraging a Vision Transformer (ViT) pretrained with self-supervision. Our proposed method takes token representations from the self-supervised ViT and leverages their correlations, via self-attention, to produce classification and segmentation predictions through separate task heads. Our model is able to effectively learn to perform classification and segmentation in the absence of pixel-level labels during training, using only image-level labels. To do this it uses attention maps, created from tokens generated by the self-supervised ViT backbone, as pixel-level pseudo-labels. We also explore a practical setup with ``mixed" supervision, where a small number of training images contains ground-truth pixel-level labels and the remaining images have only image-level labels. For this mixed setup, we propose to improve the pseudo-labels using a pseudo-label enhancer that was trained using the available ground-truth pixel-level labels. Experiments on Pascal-5i and COCO-20i demonstrate significant performance gains in a variety of supervision settings, and in particular when little-to-no pixel-level labels are available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
