arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.17874cs.CV
收藏

SmooSeg:用于无监督语义分割的平滑性先验

SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentation

Nanyang Technological University(南洋理工大学);SenseTime Research(商汤科技研究院)

Mengcheng Lan, Xinjiang Wang, Yiping Ke, Jiaxing Xu, Litong Feng, Wayne Zhang

AI总结:

本文提出SmooSeg方法,将平滑性先验引入无监督语义分割,通过平滑性损失和非对称师生预测器生成伪标签,在多个数据集上显著超越STEGO。

摘要:

无监督语义分割是一项具有挑战性的任务,旨在不依赖人工标注的情况下将图像分割成语义组。先前的工作主要侧重于利用语义一致性的先验知识或自监督学习方法中的先验概念,但这些方法往往忽略了图像分段的连贯性属性。在本文中,我们证明了平滑性先验——即度量空间中相近的特征共享相同的语义——可以通过将无监督语义分割转化为能量最小化问题来显著简化分割任务。在这一范式下,我们提出了一种名为SmooSeg的新方法,该方法利用自监督学习方法将观测之间的相近关系建模为平滑性信号。为了有效发现连贯的语义分段,我们引入了一种新颖的平滑性损失,该损失在分段内部促进分段式平滑性,同时保留不同分段之间的不连续性。此外,为了进一步提升分割质量,我们设计了一种非对称的师生式预测器,该预测器生成平滑更新的伪标签,促进观测与标签输出之间的最优拟合。得益于平滑性先验提供的丰富监督线索,我们的SmooSeg在三个数据集上的像素准确率显著优于STEGO:COCOStuff(+14.9%)、Cityscapes(+13.0%)和Potsdam-3(+5.7%)。

英文摘要:

Unsupervised semantic segmentation is a challenging task that segments images into semantic groups without manual annotation. Prior works have primarily focused on leveraging prior knowledge of semantic consistency or priori concepts from self-supervised learning methods, which often overlook the coherence property of image segments. In this paper, we demonstrate that the smoothness prior, asserting that close features in a metric space share the same semantics, can significantly simplify segmentation by casting unsupervised semantic segmentation as an energy minimization problem. Under this paradigm, we propose a novel approach called SmooSeg that harnesses self-supervised learning methods to model the closeness relationships among observations as smoothness signals. To effectively discover coherent semantic segments, we introduce a novel smoothness loss that promotes piecewise smoothness within segments while preserving discontinuities across different segments. Additionally, to further enhance segmentation quality, we design an asymmetric teacher-student style predictor that generates smoothly updated pseudo labels, facilitating an optimal fit between observations and labeling outputs. Thanks to the rich supervision cues of the smoothness prior, our SmooSeg significantly outperforms STEGO in terms of pixel accuracy on three datasets: COCOStuff (+14.9%), Cityscapes (+13.0%), and Potsdam-3 (+5.7%).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑