arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.17070cs.CV
收藏

PiCIE:基于聚类不变性和等变性的无监督语义分割

PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering

University of Texas at Austin(德克萨斯大学奥斯汀分校);Cornell University(康奈尔大学)

Jang Hyun Cho, Utkarsh Mall, Kavita Bala, Bharath Hariharan

摘要:

我们提出了一个新的框架,语义分割没有注释通过聚类。现成的聚类方法仅限于策展的、单标签的和以对象为中心的图像,而现实世界的数据主要是未策展的、多标签的和以场景为中心的。我们将聚类从图像扩展到像素,并将单独的聚类成员分配给每个图像中的不同实例。然而,仅仅依靠像素级特征相似性无法学习高级语义概念,并且过度适应低级视觉线索。我们提出了一种方法,将几何一致性作为一种归纳偏差,以学习光度和几何变化的不变性和等变性。通过我们新颖的学习目标,我们的框架可以学习高级语义概念。我们的方法PiCIE(使用不变性和等方差的像素级特征聚类)是第一种能够分割事物和材料类别的方法,无需任何超参数调整或特定于任务的预处理。我们的方法在COCO和Cityscapes上的表现大大优于现有基线,分别为+17.5 Acc.和+4.5 mIoU。我们表明,PiCIE为标准监督训练提供了更好的初始化。该代码可在https://github.com/janghyuncho/PiCIE上获得。

英文摘要:

We present a new framework for semantic segmentation without annotations via clustering. Off-the-shelf clustering methods are limited to curated, single-label, and object-centric images yet real-world data are dominantly uncurated, multi-label, and scene-centric. We extend clustering from images to pixels and assign separate cluster membership to different instances within each image. However, solely relying on pixel-wise feature similarity fails to learn high-level semantic concepts and overfits to low-level visual cues. We propose a method to incorporate geometric consistency as an inductive bias to learn invariance and equivariance for photometric and geometric variations. With our novel learning objective, our framework can learn high-level semantic concepts. Our method, PiCIE (Pixel-level feature Clustering using Invariance and Equivariance), is the first method capable of segmenting both things and stuff categories without any hyperparameter tuning or task-specific pre-processing. Our method largely outperforms existing baselines on COCO and Cityscapes with +17.5 Acc. and +4.5 mIoU. We show that PiCIE gives a better initialization for standard supervised training. The code is available at https://github.com/janghyuncho/PiCIE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑