arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.09040cs.CVcs.LG
收藏

伪标签引导的像素对比用于域自适应语义分割

Pseudolabel guided pixels contrast for domain adaptive semantic segmentation

East China University of Science and Technology(华东理工大学)

Jianzi Xiang, Cailu Wan, Zhu Cao

AI总结:

针对无监督域自适应语义分割中对比学习未考虑类内特征多样性的问题,提出PGPC框架,在两个基准任务上实现性能提升,且可增强其他UDA方法表现且不增加模型复杂度。

摘要:

语义分割对图像理解至关重要,但该过程需要大量像素级的精细标注。在现实场景中,获取这类标注的成本很高。语义分割的无监督域自适应(Unsupervised Domain Adaptation, UDA)是一种利用带标注的虚拟数据训练模型,并将其适配到无标注真实数据的技术。近期的一些研究采用对比学习——一种强大的自监督学习方法——来辅助该技术,但这些工作在使用对比学习时未考虑每个类别内部的特征多样性,导致类别预测出现误差。我们分析了这些工作的局限性,提出了名为Pseudo-label Guided Pixel Contrast(PGPC,伪标签引导像素对比)的新型框架,克服了以往方法的缺陷。我们还探究了如何在不引入伪标签噪声的前提下,利用目标图像的更多信息。我们在两个标准UDA基准上测试了所提方法,结果表明其性能优于现有方法。具体而言,在基于DAFormer的GTA5(Grand Theft Auto V)到Cityscapes、SYNTHIA到Cityscapes的任务上,我们的方法分别实现了5.1% mIoU和4.6% mIoU的相对提升。此外,我们的方法能够在不增加模型复杂度的情况下,提升其他UDA方法的性能。代码可在https://github.com/embar111/pgpc获取。

英文摘要:

Semantic segmentation is essential for comprehending images, but the process necessitates a substantial amount of detailed annotations at the pixel level. Acquiring such annotations can be costly in the real-world. Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels. Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique. However, these works do not take into account the diversity of features within each class when using contrastive learning, which leads to errors in class prediction. We analyze the limitations of these works and propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods. We also investigate how to use more information from target images without adding noise from pseudo-labels. We test our method on two standard UDA benchmarks and show that it outperforms existing methods. Specifically, we achieve relative improvements of 5.1% mIoU and 4.6% mIoU on the Grand Theft Auto V (GTA5) to Cityscapes and SYNTHIA to Cityscapes tasks based on DAFormer, respectively. Furthermore, our approach can enhance the performance of other UDA approaches without increasing model complexity. Code is available at https://github.com/embar111/pgpc

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑