面向遥感图像的视觉-语言模型纯化半监督语义分割
Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Images
AI总结：
本文提出SemiEarth模型，通过引入视觉-语言模型的伪标签纯化结构，提升遥感图像半监督语义分割的性能与可解释性。
摘要：
半监督语义分割（S4）能够从低成本的未标注图像中学习丰富的视觉知识。然而，传统S4架构都面临伪标签质量低的问题，尤其是在教师-学生框架中。我们提出了一种新的SemiEarth模型，引入了视觉-语言模型（VLMs）以解决遥感（RS）领域的S4问题。具体来说，我们发明了VLM伪标签纯化（VLM-PP）结构，用于净化教师网络的伪标签，从而实现显著的改进。尤其是在遥感图像的多类边界区域，VLM-PP模块可以显著提高教师生成的伪标签质量，从而正确引导学生模型的学习。此外，由于VLM-PP赋予VLMs开放世界能力，并且与S4架构无关，当其预测与伪标签出现不一致时，它可以纠正低置信度伪标签中的误分类。我们在多个RS数据集上进行了广泛的实验，结果表明我们的SemiEarth实现了SOTA性能。更重要的是，与以往SOTA RS S4方法不同，我们的模型不仅实现了优异的性能，还提供了良好的可解释性。代码已发布在https://github.com/wangshanwen001/SemiEarth。
英文摘要：
The semi-supervised semantic segmentation (S4) can learn rich visual knowledge from low-cost unlabeled images. However, traditional S4 architectures all face the challenge of low-quality pseudo-labels, especially for the teacher-student framework.We propose a novel SemiEarth model that introduces vision-language models (VLMs) to address the S4 issues for the remote sensing (RS) domain. Specifically, we invent a VLM pseudo-label purifying (VLM-PP) structure to purify the teacher network's pseudo-labels, achieving substantial improvements. Especially in multi-class boundary regions of RS images, the VLM-PP module can significantly improve the quality of pseudo-labels generated by the teacher, thereby correctly guiding the student model's learning. Moreover, since VLM-PP equips VLMs with open-world capabilities and is independent of the S4 architecture, it can correct mispredicted categories in low-confidence pseudo-labels whenever a discrepancy arises between its prediction and the pseudo-label. We conducted extensive experiments on multiple RS datasets, which demonstrate that our SemiEarth achieves SOTA performance. More importantly, unlike previous SOTA RS S4 methods, our model not only achieves excellent performance but also offers good interpretability. The code is released at https://github.com/wangshanwen001/SemiEarth.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
