对齐人类知识与视觉概念以实现可解释的医学图像分类
Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification
- Rutgers University（罗格斯大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出Explicd框架，融合领域知识建立诊断标准，通过视觉-语言模型对齐视觉概念与文本标准，实现可解释的医学图像分类，并在五个基准上提升性能。
AI中文摘要：
尽管可解释性在临床诊断中至关重要，但大多数深度学习模型仍然像黑箱一样运作，未能阐明其决策过程。在本研究中，我们探讨了可解释模型的开发，通过融合显式诊断标准的领域知识，使模型能够模仿人类专家的决策过程。我们引入了一个简单而有效的框架Explicd，用于可解释的、基于语言信息标准的诊断。Explicd首先从大型语言模型（LLMs）或人类专家处查询领域知识，以建立跨各种概念轴（如颜色、形状、纹理或疾病的特定模式）的诊断标准。通过利用预训练的视觉-语言模型，Explicd将这些标准作为知识锚点注入嵌入空间，从而促进医学图像中相应视觉概念的学习。最终的诊断结果基于编码的视觉概念与文本标准嵌入之间的相似度得分来确定。通过对五个医学图像分类基准的广泛评估，Explicd展示了其固有的可解释性，并相较于传统黑箱模型提升了分类性能。代码可在\url{https://github.com/yhygao/Explicd}获取。
英文摘要：
Although explainability is essential in the clinical diagnosis, most deep learning models still function as black boxes without elucidating their decision-making process. In this study, we investigate the explainable model development that can mimic the decision-making process of human experts by fusing the domain knowledge of explicit diagnostic criteria. We introduce a simple yet effective framework, Explicd, towards Explainable language-informed criteria-based diagnosis. Explicd initiates its process by querying domain knowledge from either large language models (LLMs) or human experts to establish diagnostic criteria across various concept axes (e.g., color, shape, texture, or specific patterns of diseases). By leveraging a pretrained vision-language model, Explicd injects these criteria into the embedding space as knowledge anchors, thereby facilitating the learning of corresponding visual concepts within medical images. The final diagnostic outcome is determined based on the similarity scores between the encoded visual concepts and the textual criteria embeddings. Through extensive evaluation of five medical image classification benchmarks, Explicd has demonstrated its inherent explainability and extends to improve classification performance compared to traditional black-box models. Code is available at \url{https://github.com/yhygao/Explicd}.
