一种用于无监督语义分割的轻量级聚类框架
A Lightweight Clustering Framework for Unsupervised Semantic Segmentation
AI总结：
提出轻量级聚类框架，利用自监督ViT注意力特征区分前景背景，通过多级聚类生成伪掩码，在PASCAL VOC和MS COCO上达到最先进性能。
摘要：
无监督语义分割旨在不使用标注数据的情况下，将图像中的每个像素分类到相应的类别中。由于获取标注数据集成本高昂，这一领域得到了广泛研究。尽管该领域先前的工作已显示出模型准确率的逐步提升，但大多数方法需要训练神经网络，这使得分割同样昂贵，尤其是在处理大规模数据集时。因此，我们提出了一种用于无监督语义分割的轻量级聚类框架。我们发现，自监督视觉Transformer的注意力特征表现出强烈的前景-背景可区分性。因此，可以采用聚类来有效分离前景和背景图像块。在我们的框架中，我们首先在数据集级、类别级和图像级执行多级聚类，并保持一致性。然后，提取的二值块级伪掩码经过上采样、细化，最后进行标注。此外，我们提供了对自监督视觉Transformer特征的全面分析，以及DINO和DINOv2之间的详细比较，以证明我们的主张。我们的框架在无监督语义分割中展现出巨大潜力，并在PASCAL VOC和MS COCO数据集上取得了最先进的结果。
英文摘要：
Unsupervised semantic segmentation aims to categorize each pixel in an image into a corresponding class without the use of annotated data. It is a widely researched area as obtaining labeled datasets is expensive. While previous works in the field have demonstrated a gradual improvement in model accuracy, most required neural network training. This made segmentation equally expensive, especially when dealing with large-scale datasets. We thus propose a lightweight clustering framework for unsupervised semantic segmentation. We discovered that attention features of the self-supervised Vision Transformer exhibit strong foreground-background differentiability. Therefore, clustering can be employed to effectively separate foreground and background image patches. In our framework, we first perform multilevel clustering across the Dataset-level, Category-level, and Image-level, and maintain consistency throughout. Then, the binary patch-level pseudo-masks extracted are upsampled, refined and finally labeled. Furthermore, we provide a comprehensive analysis of the self-supervised Vision Transformer features and a detailed comparison between DINO and DINOv2 to justify our claims. Our framework demonstrates great promise in unsupervised semantic segmentation and achieves state-of-the-art results on PASCAL VOC and MS COCO datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
