半监督语义分割遇上掩码建模：细粒度局部性学习在一致性正则化中至关重要
Semi-supervised Semantic Segmentation Meets Masked Modeling:Fine-grained Locality Learning Matters in Consistency Regularization
AI总结：
针对半监督语义分割中现有一致性正则化框架局部分割效果不佳的问题，提出MaskMatch框架，通过掩码建模代理任务强化细粒度局部性学习，结合多尺度集成生成可靠伪标签，在基准数据集上表现更优且具备即插即用灵活性。
摘要：
半监督语义分割旨在利用有限的标注图像和大量的未标注图像实现标注高效学习，其中由FixMatch推广的弱到强一致性正则化框架被广泛用作基准方案。尽管该方案行之有效，但我们观察到它在局部区域的分割效果难以令人满意。这可能是因为它最初源于图像分类任务，缺乏专门的机制来捕捉密集预测中优先需要的细粒度局部语义。为解决这一问题，我们提出了一个名为MaskMatch的新型框架，该框架支持细粒度局部性学习以实现更优的密集分割。在原有师生框架的基础上，我们设计了一个掩码建模代理任务，促使学生模型根据未掩码的图像块（即便仅占30%）预测分割结果，并强制这些预测结果与教师模型使用完整图像生成的伪标签保持一致。这一设计的灵感源于这样一种直觉：如果在邻域信息不足的情况下预测结果更具一致性，就说明实现了更强的细粒度局部性感知。此外，考虑到可靠的伪标签在上述局部性学习和原有一致性学习方案中的重要性，我们设计了一种多尺度集成策略，在生成伪标签时考虑不同抽象层级的上下文信息。在基准数据集上开展的大量实验证明了我们的方法相较于现有方法的优越性，以及其即插即用的灵活性。
英文摘要：
Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning, wherein the weak-to-strong consistency regularization framework, popularized by FixMatch, is widely used as a benchmark scheme. Despite its effectiveness, we observe that such scheme struggles with satisfactory segmentation for the local regions. This can be because it originally stems from the image classification task and lacks specialized mechanisms to capture fine-grained local semantics that prioritizes in dense prediction. To address this issue, we propose a novel framework called \texttt{MaskMatch}, which enables fine-grained locality learning to achieve better dense segmentation. On top of the original teacher-student framework, we design a masked modeling proxy task that encourages the student model to predict the segmentation given the unmasked image patches (even with 30\% only) and enforces the predictions to be consistent with pseudo-labels generated by the teacher model using the complete image. Such design is motivated by the intuition that if the predictions are more consistent given insufficient neighboring information, stronger fine-grained locality perception is achieved. Besides, recognizing the importance of reliable pseudo-labels in the above locality learning and the original consistency learning scheme, we design a multi-scale ensembling strategy that considers context at different levels of abstraction for pseudo-label generation. Extensive experiments on benchmark datasets demonstrate the superiority of our method against previous approaches and its plug-and-play flexibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
