基于多模态知识图谱和可靠性引导精化的病例感知医学图像分类
MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosis
- University of Science and Technology of China（科学技术大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出一种基于多模态知识图谱的病例感知推理框架，通过构建结构化诊断记忆、自适应检索相似病例、知识传播与注入机制以及置信度校准的决策精化方案，提升医学图像分类的性能和可解释性。
AI中文摘要：
深度学习为医学图像分类带来了显著进展，但现有方法大多依赖孤立的视觉证据，无法有效利用相似病例或外部知识。在临床实践中，诊断通常由相似历史病例及其相关症状支持。为了显式建模这一循证诊断过程，我们提出了一种由多模态知识图谱驱动的病例感知推理框架，用于医学图像分类。具体而言，我们构建了一个病例感知的多模态知识图谱作为结构化的诊断记忆，其中疾病、图像和症状按层次组织。给定输入图像，我们的方法自适应地从该记忆中检索相似病例，并提取相应的以病例为中心的子图。我们进一步引入了一种知识传播与注入机制，其中以图像为中心的图注意力网络将异质语义聚合为基于病例的特征，随后通过双向跨模态注意力机制将这些特征注入视觉表示以实现跨模态对齐。为了减轻噪声检索，我们设计了一种置信度校准的决策精化方案，通过联合考虑预测置信度和样本相似性来估计每个检索病例的可靠性，并重新加权其对最终预测的贡献，提供可解释的病例级证据。在多个医学影像数据集上的大量实验表明，我们的方法一致优于强基线，而消融和定性分析验证了其有效性和可解释性。代码可在 https://anonymous.4open.science/r/MKG-CARE-8B7B 获取。
英文摘要：
Medical image diagnosis has achieved significant progress with deep learning, yet existing methods often rely on isolated visual evidence and lack the ability to effectively leverage similar cases and external knowledge. In clinical practice, diagnosis is typically supported by similar historical cases and their associated symptoms. To explicitly model this evidence-based diagnostic process, we propose MKG-CARE, a framework that performs case-aware reasoning using multimodal knowledge graphs for explainable medical image diagnosis. Specifically, we construct a case-aware multimodal knowledge graph as a structured diagnostic memory, where diseases, images, and symptoms are hierarchically organized. Given an input image, MKG-CARE adaptively retrieves similar cases from this memory and extracts their corresponding case-centered subgraphs. We further introduce a knowledge propagation and injection mechanism, where an image-centric Graph Attention Network aggregates heterogeneous semantics within the retrieved case subgraphs, followed by bidirectional cross-modal attention to align and inject the aggregated case knowledge into visual representations. To mitigate retrieval noise, we design a confidence-calibrated decision refinement scheme that estimates each retrieved case's reliability from prediction confidence and sample similarity, and reweights its contribution to the final prediction for interpretable case-level evidence attribution. Extensive experiments on multiple medical imaging datasets demonstrate consistent improvements over strong baselines, while ablation and qualitative analyses validate the effectiveness and interpretability of our method. The code is available at https://github.com/lyxuan1022/MKG-CARE.
