arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.06992cs.CV
收藏

硬感知实例自适应自训练用于无监督跨域语义分割

Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation

School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Tencent Wechat AI(腾讯微信AI);School of Information and Communication Engineering, Beijing University of Posts and Telecommunications(北京邮电大学信息与通信工程学院);School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学学院)

Chuang Zhu, Kebin Liu, Wenqi Tang, Ke Mei, Jiaqi Zou, Tiejun Huang

AI总结:

针对无监督跨域语义分割,提出硬感知实例自适应自训练框架,通过实例自适应选择器生成高质量伪标签,并利用硬感知增强和区域自适应正则化提升性能,在多个基准上超越现有方法。

摘要:

标记训练数据与未标记测试数据之间的差异是当前深度学习模型面临的重大挑战。无监督域适应(UDA)试图解决这一问题。近期研究表明,自训练是解决UDA的一种有效方法。然而,现有方法在可扩展性和性能之间难以取得平衡。在本文中,我们提出了一种用于语义分割任务UDA的硬感知实例自适应自训练框架。为了有效提高伪标签的质量和多样性,我们开发了一种新颖的伪标签生成策略,并配备实例自适应选择器。我们通过巧妙设计的硬感知伪标签增强,利用图像间信息进一步丰富困难类别的伪标签。此外,我们提出了区域自适应正则化,以平滑伪标签区域并锐化非伪标签区域。对于非伪标签区域,我们还构建了一致性约束,以在模型优化过程中引入更强的监督信号。我们的方法简洁高效,易于推广到其他UDA方法。在GTA5到Cityscapes、SYNTHIA到Cityscapes以及Cityscapes到Oxford RobotCar上的实验表明,与最先进的方法相比,我们的方法具有优越的性能。我们的代码可在https://github.com/bupt-ai-cz/HIAST获取。

英文摘要:

The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models. Unsupervised domain adaptation (UDA) attempts to solve such problem. Recent works show that self-training is a powerful approach to UDA. However, existing methods have difficulty in balancing the scalability and performance. In this paper, we propose a hard-aware instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality and diversity of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. We further enrich the hard class pseudo-labels with inter-image information through a skillfully designed hard-aware pseudo-label augmentation. Besides, we propose the region-adaptive regularization to smooth the pseudo-label region and sharpen the non-pseudo-label region. For the non-pseudo-label region, consistency constraint is also constructed to introduce stronger supervision signals during model optimization. Our method is so concise and efficient that it is easy to be generalized to other UDA methods. Experiments on GTA5 to Cityscapes, SYNTHIA to Cityscapes, and Cityscapes to Oxford RobotCar demonstrate the superior performance of our approach compared with the state-of-the-art methods. Our codes are available at https://github.com/bupt-ai-cz/HIAST.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑