四象限分割视觉语言模型结合少样本适应与基于OCT学习的可解释方法用于糖尿病视网膜病变
Quadrant Segmentation VLM with Few-Shot Adaptation and OCT Learning-based Explainability Methods for Diabetic Retinopathy
AI总结：
本文提出一种基于四象限分割和少样本学习的多模态视觉语言模型，利用OCT和视网膜图像生成可解释性热图，提升糖尿病视网膜病变的诊断效果。
摘要：
糖尿病视网膜病变（DR）是全球导致视力丧失的主要原因之一，需要早期检测以保护视力。有限的医生资源常导致DR未被诊断。为了解决这个问题，AI模型利用病变分割进行可解释性；然而，手动标注病变对于临床医生来说是不现实的。医生需要一个能够解释分类理由而不是仅仅突出病变位置的模型。此外，当前模型是一维的，依赖单一成像模态进行可解释性并取得有限效果。相比之下，一个能够用自然语言识别个体DR病变的定量检测系统将克服这些限制，使筛查、治疗和研究应用多样化。为了解决这个问题，本文提出了一种新颖的多模态可解释性模型，利用具有少样本学习的VLM，通过分析视网膜四象限内的病变分布来模拟眼科医生的推理过程。该模型生成配对的Grad-CAM热图，展示OCT和视网膜图像上各个神经元权重的分布，从而在视觉上突出对DR严重程度分类起作用的区域。使用3000张视网膜图像和1000张OCT图像的数据集，这种创新方法解决了当前DR诊断的关键限制，提供了一种实用且全面的工具，以改善患者预后。
英文摘要：
Diabetic Retinopathy (DR) is a leading cause of vision loss worldwide, requiring early detection to preserve sight. Limited access to physicians often leaves DR undiagnosed. To address this, AI models utilize lesion segmentation for interpretability; however, manually annotating lesions is impractical for clinicians. Physicians require a model that explains the reasoning for classifications rather than just highlighting lesion locations. Furthermore, current models are one-dimensional, relying on a single imaging modality for explainability and achieving limited effectiveness. In contrast, a quantitative-detection system that identifies individual DR lesions in natural language would overcome these limitations, enabling diverse applications in screening, treatment, and research settings. To address this issue, this paper presents a novel multimodal explainability model utilizing a VLM with few-shot learning, which mimics an ophthalmologist's reasoning by analyzing lesion distributions within retinal quadrants for fundus images. The model generates paired Grad-CAM heatmaps, showcasing individual neuron weights across both OCT and fundus images, which visually highlight the regions contributing to DR severity classification. Using a dataset of 3,000 fundus images and 1,000 OCT images, this innovative methodology addresses key limitations in current DR diagnostics, offering a practical and comprehensive tool for improving patient outcomes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
