面向可解释语义分割的多尺度分组原型
Multi-Scale Grouped Prototypes for Interpretable Semantic Segmentation
AI总结：
提出一种利用多尺度图像表示和稀疏分组机制进行原型部分学习的可解释语义分割方法，在多个数据集上提升了模型稀疏性与可解释性，并缩小了与不可解释模型的性能差距。
摘要：
原型部分学习正成为一种使语义分割具有可解释性的有前途的方法。模型选择训练期间看到的真实图像块作为原型，并基于测试图像各部分与原型之间的相似性构建密集预测图。这提高了可解释性，因为用户可以根据原型信息检查预测输出与模型学习到的模式之间的联系。在本文中，我们提出了一种可解释语义分割的方法，利用多尺度图像表示进行原型部分学习。首先，我们引入一个原型层，在多个尺度上显式学习多样化的原型部分，从而在原型激活输出中产生多尺度表示。然后，我们提出一种稀疏分组机制，生成这些特定尺度原型部分的多尺度稀疏组。这提供了对多尺度对象表示之间交互的更深入理解，同时增强了分割模型的可解释性。在Pascal VOC、Cityscapes和ADE20K上进行的实验表明，所提出的方法增加了模型稀疏性，相比现有的基于原型的方法提高了可解释性，并缩小了与不可解释对应模型的性能差距。代码可在github.com/eceo-epfl/ScaleProtoSeg获取。
英文摘要：
Prototypical part learning is emerging as a promising approach for making semantic segmentation interpretable. The model selects real patches seen during training as prototypes and constructs the dense prediction map based on the similarity between parts of the test image and the prototypes. This improves interpretability since the user can inspect the link between the predicted output and the patterns learned by the model in terms of prototypical information. In this paper, we propose a method for interpretable semantic segmentation that leverages multi-scale image representation for prototypical part learning. First, we introduce a prototype layer that explicitly learns diverse prototypical parts at several scales, leading to multi-scale representations in the prototype activation output. Then, we propose a sparse grouping mechanism that produces multi-scale sparse groups of these scale-specific prototypical parts. This provides a deeper understanding of the interactions between multi-scale object representations while enhancing the interpretability of the segmentation model. The experiments conducted on Pascal VOC, Cityscapes, and ADE20K demonstrate that the proposed method increases model sparsity, improves interpretability over existing prototype-based methods, and narrows the performance gap with the non-interpretable counterpart models. Code is available at github.com/eceo-epfl/ScaleProtoSeg.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
