XCoOp：通过概念引导的上下文优化实现可解释的计算机辅助诊断提示学习
XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization
AI总结：
提出XCoOp框架，通过概念引导的上下文优化，将图像、可学习提示与临床概念对齐，实现可解释的提示学习，在提升诊断性能的同时提供视觉和文本解释，满足医疗XAI需求。
摘要：
利用大型视觉-语言模型（VLM）的强大表征来完成各种下游任务已引起越来越多的关注。在这一研究领域中，软提示学习已成为高效适配CLIP等VLM以完成图像分类等任务的代表性方法。然而，大多数现有的提示学习方法学习到的文本标记是不可解释的，无法满足医疗保健等高危场景中可解释人工智能（XAI）的严格可解释性要求。为解决这一问题，我们提出了一种新颖的可解释提示学习框架，该框架通过在多粒度上对齐图像、可学习提示和临床概念驱动提示的语义来利用医学知识。此外，我们的框架通过从大型语言模型中获取知识来解决缺乏有价值的概念标注的问题，并为提示提供视觉和文本解释。在有无概念标签的各种数据集上进行的大量实验和可解释性分析表明，我们的方法同时实现了优越的诊断性能、灵活性和可解释性，揭示了基础模型在促进XAI方面的有效性。代码将公开发布。
英文摘要：
Utilizing potent representations of the large vision-language models (VLMs) to accomplish various downstream tasks has attracted increasing attention. Within this research field, soft prompt learning has become a representative approach for efficiently adapting VLMs such as CLIP, to tasks like image classification. However, most existing prompt learning methods learn text tokens that are unexplainable, which cannot satisfy the stringent interpretability requirements of Explainable Artificial Intelligence (XAI) in high-stakes scenarios like healthcare. To address this issue, we propose a novel explainable prompt learning framework that leverages medical knowledge by aligning the semantics of images, learnable prompts, and clinical concept-driven prompts at multiple granularities. Moreover, our framework addresses the lack of valuable concept annotations by eliciting knowledge from large language models and offers both visual and textual explanations for the prompts. Extensive experiments and explainability analyses conducted on various datasets, with and without concept labels, demonstrate that our method simultaneously achieves superior diagnostic performance, flexibility, and interpretability, shedding light on the effectiveness of foundation models in facilitating XAI. The code will be made publically available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
