AICSD：用于语义分割的自适应类间相似性蒸馏
AICSD: Adaptive Inter-Class Similarity Distillation for Semantic Segmentation
AI总结：
本文提出类间相似性蒸馏（ICSD）和自适应损失加权（ALW）策略，通过迁移类间高阶关系并逐步降低教师影响，提升轻量级学生网络在语义分割中的精度。
摘要：
近年来，深度神经网络在计算机视觉任务中取得了显著的准确性。由于推理时间是一个关键因素，尤其是在语义分割等密集预测任务中，知识蒸馏已成为提高轻量级学生网络准确性的成功技术。现有方法往往忽视通道中以及不同类别之间的信息。为克服这些局限性，本文提出了一种名为类间相似性蒸馏（ICSD）的新方法用于知识蒸馏。所提方法通过从网络输出中独立计算每个类的类内分布，将高阶关系从教师网络迁移到学生网络。随后，利用每对类别分布之间的KL散度计算类间相似性矩阵以进行蒸馏。为进一步提高所提方法的有效性，提出了一种自适应损失加权（ALW）训练策略。与现有方法不同，ALW策略在训练过程接近结束时逐渐降低教师网络的影响，以考虑教师预测中的误差。在两个著名的语义分割数据集Cityscapes和Pascal VOC 2012上进行的大量实验，从mIoU和像素精度两方面验证了所提方法的有效性。定量和定性评估均表明，所提方法优于大多数现有的知识蒸馏方法。代码可在https://github.com/AmirMansurian/AICSD获取。
英文摘要：
In recent years, deep neural networks have achieved remarkable accuracy in computer vision tasks. With inference time being a crucial factor, particularly in dense prediction tasks such as semantic segmentation, knowledge distillation has emerged as a successful technique for improving the accuracy of lightweight student networks. The existing methods often neglect the information in channels and among different classes. To overcome these limitations, this paper proposes a novel method called Inter-Class Similarity Distillation (ICSD) for the purpose of knowledge distillation. The proposed method transfers high-order relations from the teacher network to the student network by independently computing intra-class distributions for each class from network outputs. This is followed by calculating inter-class similarity matrices for distillation using KL divergence between distributions of each pair of classes. To further improve the effectiveness of the proposed method, an Adaptive Loss Weighting (ALW) training strategy is proposed. Unlike existing methods, the ALW strategy gradually reduces the influence of the teacher network towards the end of training process to account for errors in teacher's predictions. Extensive experiments conducted on two well-known datasets for semantic segmentation, Cityscapes and Pascal VOC 2012, validate the effectiveness of the proposed method in terms of mIoU and pixel accuracy. The proposed method outperforms most of existing knowledge distillation methods as demonstrated by both quantitative and qualitative evaluations. Code is available at: https://github.com/AmirMansurian/AICSD
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
