arXiv 2309.125572023-09-22cs.CV

用于半监督语义分割的三视角知识蒸馏

Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation

School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；PengCheng Laboratory（鹏城实验室）；College of Computer Science, Zhejiang University（浙江大学计算机学院）

Ping Li, Junjie Chen, Li Yuan, Xianghua Xu, Mingli Song

发表信息：: IEEE Trans. on Multimedia, 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出TriKD半监督语义分割框架，通过三视角编码器和双频解码器结合知识蒸馏，提升特征多样性与选择效率，在精度和速度间取得良好折衷。

摘要：

为了减轻昂贵的人工标注成本，半监督语义分割利用少量标注图像和大量未标注图像来预测相同尺寸的像素级标签图。以往的方法通常采用协同训练，使用两个具有相同架构但不同初始化的卷积网络，这无法捕获足够多样的特征。这促使我们使用三训练并开发三视角编码器，利用不同架构的编码器来获取多样的特征，并利用知识蒸馏技术学习这些编码器之间的互补语义。此外，现有方法仅简单地拼接来自编码器和解码器的特征，导致特征冗余且需要大量内存开销。这启发我们设计一个双频解码器，通过将特征从空间域投影到频率域来选择重要特征，其中引入了双频通道注意力机制来建模特征重要性。因此，我们提出了一个用于半监督语义分割的三视角知识蒸馏框架，称为 TriKD，包含三视角编码器和双频解码器。在两个基准（即 Pascal VOC 2012 和 Cityscapes）上进行了大量实验，结果验证了所提方法的优越性，并在精度和推理速度之间取得了良好的折衷。

英文摘要：

To alleviate the expensive human labeling, semi-supervised semantic segmentation employs a few labeled images and an abundant of unlabeled images to predict the pixel-level label map with the same size. Previous methods often adopt co-training using two convolutional networks with the same architecture but different initialization, which fails to capture the sufficiently diverse features. This motivates us to use tri-training and develop the triple-view encoder to utilize the encoders with different architectures to derive diverse features, and exploit the knowledge distillation skill to learn the complementary semantics among these encoders. Moreover, existing methods simply concatenate the features from both encoder and decoder, resulting in redundant features that require large memory cost. This inspires us to devise a dual-frequency decoder that selects those important features by projecting the features from the spatial domain to the frequency domain, where the dual-frequency channel attention mechanism is introduced to model the feature importance. Therefore, we propose a Triple-view Knowledge Distillation framework, termed TriKD, for semi-supervised semantic segmentation, including the triple-view encoder and the dual-frequency decoder. Extensive experiments were conducted on two benchmarks, \ie, Pascal VOC 2012 and Cityscapes, whose results verify the superiority of the proposed method with a good tradeoff between precision and inference speed.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。