基于对比学习和大语言模型的可解释性黑色素瘤诊断
Explainable Melanoma Diagnosis with Contrastive Learning and LLM-based Report Generation
AI总结：
本文提出基于对比学习和大语言模型的可解释性黑色素瘤诊断框架，通过将临床标准映射到视觉Transformer空间，实现图像与临床解释的透明连接，提升模型可解释性与临床信任度。
AI中文摘要：
深度学习在黑色素瘤分类中已展现出专家级性能，使其成为临床皮肤科的强大工具。然而，模型的不透明性和缺乏可解释性仍然是临床应用的关键障碍，因为临床医生往往难以信任黑箱模型的决策过程。为解决这一差距，我们提出了一个跨模态可解释框架用于黑色素瘤（CEFM），其核心机制是对比学习，用于实现可解释性。具体而言，CEFM通过双投影头将黑色素瘤诊断的临床标准——不对称性、边界和颜色（ABC）——映射到视觉Transformer嵌入空间，从而将临床语义与视觉特征对齐。对齐的表示随后通过自然语言生成转换为结构化的文本解释，从而在原始图像数据和临床解释之间建立透明的联系。在公共数据集上的实验显示，准确率为92.79%，AUC为0.961，并在多个可解释性指标上实现了显著提升。定性分析进一步表明，所学嵌入的空间排列与临床医生应用ABC规则的方式一致，有效弥合了高性能分类与临床信任之间的差距。
英文摘要：
Deep learning has demonstrated expert-level performance in melanoma classification, positioning it as a powerful tool in clinical dermatology. However, model opacity and the lack of interpretability remain critical barriers to clinical adoption, as clinicians often struggle to trust the decision-making processes of black-box models. To address this gap, we present a Cross-modal Explainable Framework for Melanoma (CEFM) that leverages contrastive learning as the core mechanism for achieving interpretability. Specifically, CEFM maps clinical criteria for melanoma diagnosis-namely Asymmetry, Border, and Color (ABC)-into the Vision Transformer embedding space using dual projection heads, thereby aligning clinical semantics with visual features. The aligned representations are subsequently translated into structured textual explanations via natural language generation, creating a transparent link between raw image data and clinical interpretation. Experiments on public datasets demonstrate 92.79% accuracy and an AUC of 0.961, along with significant improvements across multiple interpretability metrics. Qualitative analyses further show that the spatial arrangement of the learned embeddings aligns with clinicians' application of the ABC rule, effectively bridging the gap between high-performance classification and clinical trust.
