利用主掩码提议提升无监督语义分割性能
Boosting Unsupervised Semantic Segmentation with Principal Mask Proposals
AI总结：
该研究提出PriMaPs主掩码提议及PriMaPs-EM算法，基于自监督预训练模型实现无监督语义分割，在多模型多数据集上取得有竞争力的结果，还可正交提升现有SOTA流水线性能。
摘要：
无监督语义分割旨在无需任何标注的前提下，通过识别图像语料库中的全局语义类别，自动将图像划分为具有语义意义的区域。基于自监督表征学习的最新进展，我们聚焦于如何利用这些大型预训练模型完成无监督分割下游任务。我们提出PriMaPs（主掩码提议）方法，可根据图像的特征表征将其分解为具有语义意义的掩码。在此基础上，我们通过随机期望最大化算法PriMaPs-EM将类别原型拟合到PriMaPs上，实现无监督语义分割。尽管PriMaPs-EM概念简单，但在DINO、DINOv2等多种预训练骨干模型，以及Cityscapes、COCO-Stuff、Potsdam-3等不同数据集上均取得了具有竞争力的结果。重要的是，当与当前最先进的无监督语义分割流水线正交结合应用时，PriMaPs-EM能够提升现有结果。代码可在https://github.com/visinf/primaps获取。
英文摘要：
Unsupervised semantic segmentation aims to automatically partition images into semantically meaningful regions by identifying global semantic categories within an image corpus without any form of annotation. Building upon recent advances in self-supervised representation learning, we focus on how to leverage these large pre-trained models for the downstream task of unsupervised segmentation. We present PriMaPs - Principal Mask Proposals - decomposing images into semantically meaningful masks based on their feature representation. This allows us to realize unsupervised semantic segmentation by fitting class prototypes to PriMaPs with a stochastic expectation-maximization algorithm, PriMaPs-EM. Despite its conceptual simplicity, PriMaPs-EM leads to competitive results across various pre-trained backbone models, including DINO and DINOv2, and across different datasets, such as Cityscapes, COCO-Stuff, and Potsdam-3. Importantly, PriMaPs-EM is able to boost results when applied orthogonally to current state-of-the-art unsupervised semantic segmentation pipelines. Code is available at https://github.com/visinf/primaps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
