从像素到解释：结合CNN-Transformer集成、视觉可解释性和视觉语言模型的可解释性糖尿病视网膜病变分级
From Pixels to Explanations: Interpretable Diabetic Retinopathy Grading with CNN-Transformer Ensembles, Visual Explainability and Vision-Language Models
AI总结：
本文提出结合强判别模型与多模态解释的方法，利用APTOS 2019基准评估六种CNN和Transformer模型，通过集成策略提升分级一致性，并利用视觉语言模型生成可解释的视网膜病变分级结果。
摘要：
糖尿病视网膜病变（DR）筛查的质量依赖于正确分级严重程度的能力；然而，许多深度学习（DL）分类器在临床环境中难以解释。本研究提出一种结合强判别模型与多模态解释的方法，将视网膜像素转化为临床可解释的输出。使用APTOS 2019基准，我们在受控协议下评估了六个代表性的CNN和Transformer后端，在分层五折交叉验证中进行评估。随后，我们比较了集成策略（硬投票、加权软投票、堆叠）并研究了一种混合类级融合变体以利用等级特定优势。在可解释性方面，我们利用视觉语言模型（VLMs）生成Grad-CAM++视觉属性图和简短的文本解释，这些模型在视网膜图像和分类器输出条件下受保守提示约束。现代CNN后端（ResNet-50和ConvNeXt-Tiny）提供了最强的单模型基线，交叉验证QWK分别达到0.919和0.914。集成提高了顺序一致性，加权软投票在所有折叠中最为一致（QWK 0.934 ± 0.017）。混合类级融合具有竞争力，但在配对折叠比较中并未在标准融合上产生统计上可靠的改进（Holm调整后的p ≥ 1.000）。在解释质量方面，Grad-CAM++提供了合理但粗略的定位，而VLM解释通常与等级一致。定量上，VLM变体在临床完整性和模板级语义相似性之间存在权衡（覆盖0.700 vs. BERTScore 0.072），而图像-文本对齐相当（CLIPScore大约0.34）
英文摘要：
The quality of diabetic retinopathy (DR) screening relies on the ability to correctly grade severity; however, many deep-learning (DL) classifiers cannot be easily interpreted in the clinical context. This study presents a methodology that combines strong discriminative models with multimodal explanations, converting retinal pixels into clinically interpretable outputs. Using the APTOS 2019 benchmark, we evaluated six representative CNN- and transformer-based backbones under a controlled protocol with stratified five-fold cross-validation. We then compared ensembling strategies (hard voting, weighted soft voting, stacking) and investigated a hybrid class-level fusion variant to exploit grade-specific advantages. For interpretability, we produced Grad-CAM++ visual attribution maps and short textual rationales using vision-language models (VLMs) conditioned on the fundus image and classifier outputs under conservative prompting constraints. Modern CNN backbones (ResNet-50 and ConvNeXt-Tiny) provided the strongest single-model baselines, with cross-validated QWK up to 0.919 and 0.914, respectively. Ensembling improved ordinal agreement, and weighted soft voting was the most consistent across folds (QWK 0.934 +/- 0.017). Hybrid class-level fusion was competitive but did not yield a statistically reliable improvement over standard fusion in paired fold comparisons (Holm-adjusted p >= 1.000). For explanation quality, Grad-CAM++ offered plausible but coarse localization, and VLM rationales were generally grade-consistent. Quantitatively, VLM variants showed a trade-off between clinical completeness and template-level semantic similarity (coverage 0.700 vs. BERTScore 0.072), while image-text alignment was comparable (CLIPScore approximately 0.34).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
