arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.02403q-bio.QMcs.AIcs.CV
收藏

基于微调多模态大语言模型的青光眼检测与结构化OCT报告生成

Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model

University of California San Diego（加州大学圣迭戈分校）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Jalil Jalili, Yashraj Gavhane, Evan Walker, Anna Heinke, Christopher Bowd, Akram Belghith, Massimo A. Fazio, Christopher A. Girkin, C. Gustavo De Moraes, Jeffre… 展开作者

Jalil Jalili, Yashraj Gavhane, Evan Walker, Anna Heinke, Christopher Bowd, Akram Belghith, Massimo A. Fazio, Christopher A. Girkin, C. Gustavo De Moraes, Jeffrey M. Liebmann, Sally L. Baxter, Robert N. Weinreb, Linda M. Zangwill, Mark Christopher

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究对Llama 3.2 Vision-Instruct多模态大模型进行微调，基于OCT影像实现图像质量分诊、青光眼检测及分区RNFL变薄评估，并生成高质量结构化临床报告，可为临床OCT评估提供支持。

摘要：

目的：开发一种可解释的多模态大语言模型（MM-LLM），该模型需实现两项功能：（1）对视神经头（ONH）OCT环形扫描图像进行质量筛查；（2）生成包含青光眼诊断及分区视网膜神经纤维层（RNFL）变薄评估的结构化临床报告。研究设计：回顾性队列研究，纳入来自DIGS和ADAGES队列的1310名受试者，共43849张Spectralis ONH OCT环形扫描图像（对应1331只青光眼患眼、867只健康眼）。方法：对Llama 3.2 Vision-Instruct这一多模态大语言模型进行微调，使其能够生成OCT影像数据的临床描述。训练数据由成对的OCT图像与自动生成的结构化临床报告组成，报告内容涵盖整体及分区的RNFL变薄情况。质量不佳的扫描图像被标记为不可用，并与固定的拒绝说明配对。研究在留出的测试集上评估模型的三项任务性能：质量评估、青光眼检测，以及七个解剖分区的RNFL变薄分类，评估指标包括准确率、灵敏度、特异度、精确率和F1值；同时采用标准文本评估指标衡量模型生成报告的质量。结果：模型在质量分诊任务上达到0.90的准确率和0.98的特异度；青光眼检测任务的准确率为0.86（灵敏度0.91、特异度0.73、F1值0.91）；RNFL变薄预测的准确率在0.83到0.94之间，其中整体分区和颞侧分区的表现最优。文本生成得分显示其与参考报告高度一致（BLEU：0.82；ROUGE-1：0.94；ROUGE-2：0.87；ROUGE-L：0.92；BERTScore-F1：0.99）。结论：经微调的多模态大语言模型可基于OCT影像生成准确的临床描述，在识别图像质量问题和检测青光眼方面均达到较高准确率，同时还能提供RNFL变薄的分区描述，可为临床OCT评估提供支持。

英文摘要：

Objective: To develop an explainable multimodal large language model (MM-LLM) that (1) screens optic nerve head (ONH) OCT circle scans for quality and (2) generates structured clinical reports that include glaucoma diagnosis and sector-wise retinal nerve fiber layer (RNFL) thinning assessments. Design: Retrospective cohort study of 1,310 subjects contributing 43,849 Spectralis ONH OCT circle scans (1,331 glaucomatous and 867 healthy eyes) from the DIGS and ADAGES cohorts. Methods: A MM-LLM (Llama 3.2 Vision-Instruct model) was fine-tuned to generate clinical descriptions of OCT imaging data. Training data included paired OCT images and automatically generated, structured clinical reports that described global and sectoral RNFL thinning. Poor-quality scans were labeled as unusable and paired with a fixed refusal statement. The model was evaluated on a held-out test set for three tasks: quality assessment, glaucoma detection, and RNFL thinning classification across seven anatomical sectors. Evaluation metrics included accuracy, sensitivity, specificity, precision, and F1-score. Model description quality was also evaluated using standard text evaluation metrics. Results: The model achieved 0.90 accuracy and 0.98 specificity for quality triage. For glaucoma detection, accuracy was 0.86 (sensitivity 0.91, specificity 0.73, F1-score 0.91). RNFL thinning prediction accuracy ranged from 0.83 to 0.94, with highest performance in global and temporal sectors. Text generation scores showed strong alignment with reference reports (BLEU: 0.82; ROUGE-1: 0.94; ROUGE-2: 0.87; ROUGE-L: 0.92; BERTScore-F1: 0.99). Conclusions: The fine-tuned MM-LLM generated accurate clinical descriptions based on OCT imaging. The model achieved high accuracy in identifying image quality issues and detecting glaucoma. The model also provided sectoral descriptions of RNFL thinning to help support clinical OCT evaluation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑