ProtoPFormer: 在视觉变换器中聚焦原型部分以实现可解释的图像识别
ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition
AI总结：
ProtoPFormer通过引入全局和局部原型，提升视觉变换器在可解释图像识别中的性能和可解释性。
摘要：
原型部分网络（ProtoPNet）因其对可解释人工智能（XAI）的自解释性而受到广泛关注，并推动了许多后续研究。然而，当直接将ProtoPNet应用于视觉变换器（ViT）主干时，学习到的原型存在“干扰”问题：它们有较高的被背景激活的概率，而对前景关注较少。由于变换器模型强大的长期依赖建模能力，基于变换器的ProtoPNet难以聚焦原型部分，从而严重损害了其固有的可解释性。本文提出原型部分变换器（ProtoPFormer），以适当且有效地将基于原型的方法与ViT结合，用于可解释的图像识别。所提出的方法引入了全局和局部原型，根据ViTs的架构特性，用于捕捉和突出目标的代表性整体和局部特征。全局原型用于提供对象的全局视图，以指导局部原型专注于前景，同时消除背景的影响。之后，局部原型被显式监督以专注于各自原型的视觉部分，从而提高整体可解释性。广泛的实验表明，所提出的全局和局部原型可以相互纠正并共同做出最终决策，这使决策过程能够从整体和局部视角分别忠实且透明地进行推理。此外，ProtoPFormer在最先进的（SOTA）原型基线方法上始终实现了优越的性能和可视化结果。我们的代码已发布在https://github.com/zju-vipa/ProtoPFormer。
英文摘要：
Prototypical part network (ProtoPNet) has drawn wide attention and boosted many follow-up studies due to its self-explanatory property for explainable artificial intelligence (XAI). However, when directly applying ProtoPNet on vision transformer (ViT) backbones, learned prototypes have a "distraction" problem: they have a relatively high probability of being activated by the background and pay less attention to the foreground. The powerful capability of modeling long-term dependency makes the transformer-based ProtoPNet hard to focus on prototypical parts, thus severely impairing its inherent interpretability. This paper proposes prototypical part transformer (ProtoPFormer) for appropriately and effectively applying the prototype-based method with ViTs for interpretable image recognition. The proposed method introduces global and local prototypes for capturing and highlighting the representative holistic and partial features of targets according to the architectural characteristics of ViTs. The global prototypes are adopted to provide the global view of objects to guide local prototypes to concentrate on the foreground while eliminating the influence of the background. Afterwards, local prototypes are explicitly supervised to concentrate on their respective prototypical visual parts, increasing the overall interpretability. Extensive experiments demonstrate that our proposed global and local prototypes can mutually correct each other and jointly make final decisions, which faithfully and transparently reason the decision-making processes associatively from the whole and local perspectives, respectively. Moreover, ProtoPFormer consistently achieves superior performance and visualization results over the state-of-the-art (SOTA) prototype-based baselines. Our code has been released at https://github.com/zju-vipa/ProtoPFormer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
