用于缓解医学图像中捷径学习的凝视引导视觉图神经网络

中文摘要 AI 辅助

深度神经网络在医学图像分析中已展现出卓越性能。然而，其因捷径学习而对虚假相关性产生的敏感性引发了人们对网络可解释性和可靠性的担忧。此外，在疾病指标往往细微且稀疏的医学场景中，捷径学习问题会更加严重。本文提出了一种新颖的凝视引导视觉图神经网络（称为GD-ViG），以利用放射科医生凝视中的视觉模式作为专家知识，引导网络关注疾病相关区域，从而缓解捷径学习。GD-ViG由一个凝视图生成器（GMG）和一个凝视引导分类器（GDC）组成。GMG将图神经网络的全局建模能力与卷积神经网络的局部性相结合，基于放射科医生的视觉模式生成凝视图。值得注意的是，它在推理阶段无需真实凝视数据，从而增强了网络的实际适用性。利用凝视作为专家知识，GDC通过同时结合特征距离和凝视距离来指导图结构的构建，使网络能够聚焦于疾病相关的前景区域，从而避免捷径学习并提高网络的可解释性。在两个公开医学图像数据集上的实验表明，GD-ViG优于最先进的方法，并有效缓解了捷径学习。我们的代码可在https://github.com/SX-SS/GD-ViG获取。

英文摘要

Deep neural networks have demonstrated remarkable performance in medical image analysis. However, its susceptibility to spurious correlations due to shortcut learning raises concerns about network interpretability and reliability. Furthermore, shortcut learning is exacerbated in medical contexts where disease indicators are often subtle and sparse. In this paper, we propose a novel gaze-directed Vision GNN (called GD-ViG) to leverage the visual patterns of radiologists from gaze as expert knowledge, directing the network toward disease-relevant regions, and thereby mitigating shortcut learning. GD-ViG consists of a gaze map generator (GMG) and a gaze-directed classifier (GDC). Combining the global modelling ability of GNNs with the locality of CNNs, GMG generates the gaze map based on radiologists' visual patterns. Notably, it eliminates the need for real gaze data during inference, enhancing the network's practical applicability. Utilizing gaze as the expert knowledge, the GDC directs the construction of graph structures by incorporating both feature distances and gaze distances, enabling the network to focus on disease-relevant foregrounds. Thereby avoiding shortcut learning and improving the network's interpretability. The experiments on two public medical image datasets demonstrate that GD-ViG outperforms the state-of-the-art methods, and effectively mitigates shortcut learning. Our code is available at https://github.com/SX-SS/GD-ViG.

发表机构

• School of Information Science and Technology, Northwest University（西北大学信息科学与技术学院）
• Northwestern University（西北大学）

机构由 AI 辅助整理，请以论文原文为准。