评估基于Transformer的医学影像中注意力图的可视化解释
Evaluating Visual Explanations of Attention Maps for Transformer-based Medical Imaging
AI总结：
本研究在四个医学影像数据集上系统比较了注意力图与其他可解释性方法在视觉Transformer中的表现，发现注意力图虽优于GradCAM但不及Transformer专用方法，其可解释性效果具有情境依赖性。
摘要：
尽管视觉Transformer（ViTs）近期在医学影像问题上展现出卓越性能，但它们面临与卷积神经网络等先前架构类似的可解释性问题。近期研究表明，作为ViTs决策过程一部分的注意力图，有望通过识别影响预测的区域来解决可解释性问题，尤其是在采用自监督学习预训练的模型中。本研究将注意力图的可视化解释与医学影像问题中其他常用方法进行比较。为此，我们采用四个不同的医学影像数据集，涉及（1）结肠息肉、（2）乳腺肿瘤、（3）食管炎症以及（4）骨折与内固定植入物的识别。通过在上述数据集上使用多种监督和自监督预训练的ViTs进行大规模实验，我们发现尽管注意力图在特定条件下展现出潜力且总体上在可解释性方面优于GradCAM，但它们仍被Transformer专用的可解释性方法所超越。我们的发现表明，注意力图作为一种可解释性方法的有效性依赖于具体情境，并且可能具有局限性，因为它们无法始终如一地提供稳健医学决策所需的全面洞察。
英文摘要：
Although Vision Transformers (ViTs) have recently demonstrated superior performance in medical imaging problems, they face explainability issues similar to previous architectures such as convolutional neural networks. Recent research efforts suggest that attention maps, which are part of decision-making process of ViTs can potentially address the explainability issue by identifying regions influencing predictions, especially in models pretrained with self-supervised learning. In this work, we compare the visual explanations of attention maps to other commonly used methods for medical imaging problems. To do so, we employ four distinct medical imaging datasets that involve the identification of (1) colonic polyps, (2) breast tumors, (3) esophageal inflammation, and (4) bone fractures and hardware implants. Through large-scale experiments on the aforementioned datasets using various supervised and self-supervised pretrained ViTs, we find that although attention maps show promise under certain conditions and generally surpass GradCAM in explainability, they are outperformed by transformer-specific interpretability methods. Our findings indicate that the efficacy of attention maps as a method of interpretability is context-dependent and may be limited as they do not consistently provide the comprehensive insights required for robust medical decision-making.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
