R-Cut：通过关系加权输出与裁剪增强视觉Transformer的可解释性
R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut
AI总结：
针对Transformer图像分类模型的可解释性问题，本文提出包含关系加权输出与裁剪模块的R-Cut方法，生成类别专属可视化图，在ImageNet和LRN数据集上表现优于现有方法，且各模块有效性经消融实验验证。
摘要：
基于Transformer的模型已在自然语言处理（NLP）领域广泛流行，还被大量应用于计算机视觉任务以及GPT4等多模态模型中。本文提出一种增强基于Transformer的图像分类模型可解释性的新方法。我们的方法旨在通过提供类别专属图的可视化结果，提升分类结果的可信度，同时帮助用户更深入地理解模型，以支撑下游任务。我们引入了两个模块：“关系加权输出（Relationship Weighted Out）”模块和“裁剪（Cut）”模块。“关系加权输出”模块专注于从中间层提取类别专属信息，使我们能够高亮相关特征。此外，“裁剪”模块会执行细粒度特征分解，同时考虑位置、纹理、颜色等因素。通过整合这两个模块，我们生成了稠密的类别专属可视化可解释性图。我们在ImageNet数据集上开展了大量定性与定量实验，验证了方法的有效性。此外，我们还在专门为自动驾驶危险预警设计的LRN数据集上进行了大量实验，以评估我们的方法在复杂背景下的可解释性。结果表明，该方法相比以往方法有显著提升。我们还开展了消融实验，验证每个模块的有效性。通过这些实验，我们确认了每个模块各自的贡献，从而夯实了所提方法的整体有效性。
英文摘要：
Transformer-based models have gained popularity in the field of natural language processing (NLP) and are extensively utilized in computer vision tasks and multi-modal models such as GPT4. This paper presents a novel method to enhance the explainability of Transformer-based image classification models. Our method aims to improve trust in classification results and empower users to gain a deeper understanding of the model for downstream tasks by providing visualizations of class-specific maps. We introduce two modules: the ``Relationship Weighted Out" and the ``Cut" modules. The ``Relationship Weighted Out" module focuses on extracting class-specific information from intermediate layers, enabling us to highlight relevant features. Additionally, the ``Cut" module performs fine-grained feature decomposition, taking into account factors such as position, texture, and color. By integrating these modules, we generate dense class-specific visual explainability maps. We validate our method with extensive qualitative and quantitative experiments on the ImageNet dataset. Furthermore, we conduct a large number of experiments on the LRN dataset, specifically designed for automatic driving danger alerts, to evaluate the explainability of our method in complex backgrounds. The results demonstrate a significant improvement over previous methods. Moreover, we conduct ablation experiments to validate the effectiveness of each module. Through these experiments, we are able to confirm the respective contributions of each module, thus solidifying the overall effectiveness of our proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
