arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.06331cs.CV
收藏

面向跨域分割的语义连通性驱动伪标签

Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation

Xidian University(西安电子科技大学);University of Trento(特伦托大学);University of Nottingham(诺丁汉大学)

Dong Zhao, Ruizhi Yang, Shuang Wang, Qi Zang, Yang Hu, Licheng Jiao, Nicu Sebe, Zhun Zhong

AI总结:

针对跨域语义分割自训练中散斑伪标签难学且噪声难纠正的问题,提出SeCo,借助SAM聚合语义连通体并依据损失分布纠正连通噪声,显著提升多类域适应任务性能。

摘要:

目前,自训练是跨域语义分割中的主流方法,通过使用被赋予可靠伪标签的像素进行训练来提升模型效果。然而,我们发现该范式存在两个关键局限。(1)大多数可靠像素呈现散斑状模式,且主要位于语义区域中心。这给模型准确学习语义带来挑战。(2)散斑像素中的类别噪声难以定位和纠正,导致自训练中的误差累积。为解决这些局限,我们提出了一种名为语义连通性驱动伪标签(Semantic Connectivity-driven pseudo-labeling,SeCo)的新方法。该方法在连通性层面构建伪标签,因此能够促进结构化、低噪声语义的学习。具体而言,SeCo包含两个关键组件:像素语义聚合(Pixel Semantic Aggregation,PSA)和语义连通性纠正(Semantic Connectivity Correction,SCC)。首先,PSA将语义划分为“stuff”和“things”类别,并通过与Segment Anything Model(SAM)的高效交互,将散斑状伪标签聚合为语义连通体。这不仅使我们能够获得准确边界,还简化了噪声定位。随后,SCC引入一个简单的连通性分类任务,使其能够在损失分布的引导下定位并纠正连通性噪声。大量实验表明,SeCo可灵活应用于各种跨域语义分割任务,包括传统的无监督、源域无关和黑盒域适应,并显著提升现有最先进方法的性能。代码可在https://github.com/DZhaoXd/SeCo获取。

英文摘要:

Presently, self-training stands as a prevailing approach in cross-domain semantic segmentation, enhancing model efficacy by training with pixels assigned with reliable pseudo-labels. However, we find two critical limitations in this paradigm. (1) The majority of reliable pixels exhibit a speckle-shaped pattern and are primarily located in the central semantic region. This presents challenges for the model in accurately learning semantics. (2) Category noise in speckle pixels is difficult to locate and correct, leading to error accumulation in self-training. To address these limitations, we propose a novel approach called Semantic Connectivity-driven pseudo-labeling (SeCo). This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics. Specifically, SeCo comprises two key components: Pixel Semantic Aggregation (PSA) and Semantic Connectivity Correction (SCC). Initially, PSA divides semantics into 'stuff' and 'things' categories and aggregates speckled pseudo-labels into semantic connectivity through efficient interaction with the Segment Anything Model (SAM). This enables us not only to obtain accurate boundaries but also simplifies noise localization. Subsequently, SCC introduces a simple connectivity classification task, which enables locating and correcting connectivity noise with the guidance of loss distribution. Extensive experiments demonstrate that SeCo can be flexibly applied to various cross-domain semantic segmentation tasks, including traditional unsupervised, source-free, and black-box domain adaptation, significantly improving the performance of existing state-of-the-art methods. The code is available at https://github.com/DZhaoXd/SeCo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑