arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.15212eess.IV
收藏

用于视网膜图像眼部疾病筛查的上下文感知视觉语言基础模型

Context-Aware Vision Language Foundation Models for Ocular Disease Screening in Retinal Images

Lucie Berger, Mathieu Lamard, Philippe Zhang, Laurent Borderie, Alexandre Le Guilcher, Pascale Massin, Béatrice Cochener, Gwenolé Quellec, Sarah Matta

备注：
4 pages
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究基于FLAIR模型提出上下文感知视觉语言基础模型，利用OPHDIAT数据集的多维度临床与图像数据提升眼部疾病筛查的域内性能与域外泛化能力。

摘要：

基础模型是在海量多样数据上训练的大规模通用系统，旨在学习可泛化的表征。由于医学成像领域的数据变异性和域偏移是主要挑战，基础模型仅需少量微调即可适配的特性使其在该领域极具应用前景。目前文献中主要有两类基础模型：自监督模型以及更近期出现的视觉语言模型。本研究推进了视觉语言基础（VLF）模型在眼部疾病筛查中的应用，所用数据集为OPHDIAT数据集，该数据集包含来自法国糖尿病视网膜病变（DR）筛查网络的近70万张眼底照片。该数据集提供了丰富的临床数据（患者特定信息，如糖尿病健康状况和治疗情况）、标注好的诊断结果、眼科医生的文本形式检查发现，以及每次检查对应的多张视网膜图像。我们以FLAIR模型——一种用于视网膜病变分类的VLF模型——为基础，提出了新型的上下文感知VLF模型（例如联合分析同一次就诊的多张图像，或利用过往诊断结果和上下文数据），以充分挖掘OPHDIAT数据集的丰富信息，并增强模型对域偏移的鲁棒性。我们在域内数据（OPHDIAT的测试子集）和域外数据（公开数据集）上对所提方法进行了评估，以衡量其泛化性能。我们的模型在DR分级的域内性能上有所提升，曲线下面积（AUC）范围为0.851至0.9999，并且在域外数据的眼部疾病检测任务中泛化表现良好（AUC：0.631-0.913）。

英文摘要：

Foundation models are large-scale versatile systems trained on vast quantities of diverse data to learn generalizable representations. Their adaptability with minimal fine-tuning makes them particularly promising for medical imaging, where data variability and domain shifts are major challenges. Currently, two types of foundation models dominate the literature: self-supervised models and more recent vision-language models. In this study, we advance the application of vision-language foundation (VLF) models for ocular disease screening using the OPHDIAT dataset, which includes nearly 700,000 fundus photographs from a French diabetic retinopathy (DR) screening network. This dataset provides extensive clinical data (patient-specific information such as diabetic health conditions, and treatments), labeled diagnostics, ophthalmologists text-based findings, and multiple retinal images for each examination. Building on the FLAIR model $\unicode{x2013}$ a VLF model for retinal pathology classification $\unicode{x2013}$ we propose novel context-aware VLF models (e.g jointly analyzing multiple images from the same visit or taking advantage of past diagnoses and contextual data) to fully leverage the richness of the OPHDIAT dataset and enhance robustness to domain shifts. Our approaches were evaluated on both in-domain (a testing subset of OPHDIAT) and out-of-domain data (public datasets) to assess their generalization performance. Our model demonstrated improved in-domain performance for DR grading, achieving an area under the curve (AUC) ranging from 0.851 to 0.9999, and generalized well to ocular disease detection on out-of-domain data (AUC: 0.631-0.913).

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑