通过带特征记忆库的统一流缩小标注数据与未标注数据之间的差距

中文摘要 AI 辅助

尽管半监督语义分割（S⁴）利用丰富的未标注数据减少人工标注负担，但标注数据与未标注数据的独立训练会导致前者占据主导地位，严重降低伪标签质量。为解决这一挑战，我们提出一种带特征记忆库（UFFM）的新型遥感（RS）半监督语义分割方法。具体而言，UFFM包含两项关键创新：统一流（UF）与特征记忆库（FMB）。统一流是一种新型训练流，通过结合外部视觉基础模型（VFM）与遥感领域教师生成偏差更小的伪标签，并在统一训练目标下联合优化标注数据与伪标注数据。特征记忆库是半监督语义分割的新型记忆模块，在训练过程中动态更新类别特定特征，并通过类别特征对齐缩小标注数据与未标注数据之间的特征差异。为验证模型有效性，我们在遥感数据集上开展了大量实验，实验结果表明，我们的方法优于当前最优（SOTA）半监督语义分割方法，且结果证明了我们的贡献在缩小标注数据与未标注数据之间的优化及特征表示差距方面的有效性。我们的代码已发布在该链接：this https URL。

英文摘要

Although semi-supervised semantic segmentation ($\text{S}^4$) utilizes abundant unlabeled data to reduce manual labeling burdens, independent training of labeled and unlabeled data causes the former to dominate, which severely degrades pseudo-label quality. To address this challenges, we propose a novel remote sensing (RS) $\text{S}^4$ method via unified flow with feature memory bank (UFFM). Specifically, UFFM comprises two key innovations: unified flow (UF) and feature memory bank (FMB). The UF is a new training flow that generates less biased pseudo-labels by combining an external visual foundation model (VFM) with an RS domain teacher, and jointly optimizes labeled and pseudo-labeled data under a unified training objective. The FMB is a novel memory module for $\text{S}^4$ that dynamically updates class-specific features during training and reduces the feature discrepancy between labeled and unlabeled data through class-feature alignment. To verify the effectiveness of our model, we conduct extensive experiments on RS datasets. The experimental results show the superiority of our method over SOTA $\text{S}^4$ methods. Moreover, the results demonstrate the effectiveness of our contributions in bridging the optimization and feature representation gap between labeled and unlabeled data. Our code is released at \href{https://github.com/wangshanwen001/RS-UFFM}{https://github.com/wangshanwen001/RS-UFFM}.

发表机构

• City University of Macau（澳门城市大学）
• Southeast University（东南大学）
• Ocean University of China（中国海洋大学）
• Nantes Université（南特大学）
• Ecole Centrale Nantes（南特中央理工学院）
• CAPACITES SAS
• CNRS（法国国家科学研究中心）
• LS2N（南特数字科学实验室）

机构由 AI 辅助整理，请以论文原文为准。