arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.08808cs.CV
收藏

SePiCo:语义引导的像素对比度域自适应语义分割

SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation

Beijing Institute of Technology(北京理工大学);Tsinghua University(清华大学)

Binhui Xie, Shuang Li, Mingjia Li, Chi Harold Liu, Gao Huang, Guoren Wang

摘要:

域自适应语义分割试图通过利用在标记源域上训练的监督模型对未标记目标域进行令人满意的密集预测。在这项工作中,我们提出了语义引导的像素对比度(SePiCo),这是一种新的单阶段自适应框架,它突出了单个像素的语义概念,以促进跨域的类区分和类平衡像素表示的学习,最终提高自训练方法的性能。具体来说,为了探索适当的语义概念,我们首先研究了一种质心感知像素对比度,该像素对比度采用整个源域或单个源图像的类别质心来指导判别特征的学习。考虑到语义概念中可能缺乏类别多样性,我们开辟了一条包含足够数量实例的分布视角的道路,即分布感知像素对比度,其中我们从标记源数据的统计数据中近似每个语义类别的真实分布。此外,这样的优化目标可以推导出一个封闭形式的上限隐含地涉及无限数量的(不)相似的对,使其计算效率。大量的实验表明,SePiCo不仅有助于稳定训练,而且还能产生有区别的表示,在合成到真实和白天到夜间的适应场景中都取得了重大进展。

英文摘要:

Domain adaptive semantic segmentation attempts to make satisfactory dense predictions on an unlabeled target domain by utilizing the supervised model trained on a labeled source domain. In this work, we propose Semantic-Guided Pixel Contrast (SePiCo), a novel one-stage adaptation framework that highlights the semantic concepts of individual pixels to promote learning of class-discriminative and class-balanced pixel representations across domains, eventually boosting the performance of self-training methods. Specifically, to explore proper semantic concepts, we first investigate a centroid-aware pixel contrast that employs the category centroids of the entire source domain or a single source image to guide the learning of discriminative features. Considering the possible lack of category diversity in semantic concepts, we then blaze a trail of distributional perspective to involve a sufficient quantity of instances, namely distribution-aware pixel contrast, in which we approximate the true distribution of each semantic category from the statistics of labeled source data. Moreover, such an optimization objective can derive a closed-form upper bound by implicitly involving an infinite number of (dis)similar pairs, making it computationally efficient. Extensive experiments show that SePiCo not only helps stabilize training but also yields discriminative representations, making significant progress on both synthetic-to-real and daytime-to-nighttime adaptation scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑