用于术前多模态数据精确全面脑肿瘤诊断的视觉-语言基础模型
A Vision-Language Foundation Model for Precise and Comprehensive Brain Tumor Diagnosis from Preoperative Multimodal Data
- The University of Hong Kong（香港大学）
- Xiangya Hospital, Central South University（中南大学湘雅医院）
- Changde Hospital, Xiangya School of Medicine, Central South University (The First People’s Hospital of Changde City)（中南大学湘雅医学院常德医院（常德市第一人民医院））
- Stanford University（斯坦福大学）
- The Second Affiliated Hospital, Hengyang Medical School, University of South China（南华大学衡阳医学院附属第二医院）
- The Second Affiliated Hospital, Jiangxi Medical College, Nanchang University（南昌大学江西医学院第二附属医院）
- The First Affiliated Hospital, Jiangxi Medical College, Nanchang University（南昌大学江西医学院第一附属医院）
- Jiangxi Provincial People’s Hospital, The First Affiliated Hospital of Nanchang Medical College（南昌医学院第一附属医院江西省人民医院）
- The Affiliated Children’s Hospital of Xiangya School of Medicine, Hunan Children’s Hospital（湖南儿童医院湘雅医学院附属儿童医院）
- Shenzhen Second People’s Hospital（深圳市第二人民医院）
- First Hospital of Lanzhou University（兰州大学第一医院）
- The Third Xiangya Hospital, Central South University（中南大学湘雅三医院）
- Tongji Hospital, School of Medicine, Tongji University（同济大学附属同济医院）
- Chongqing Traditional Chinese Medicine Hospital（重庆市中医院）
- Basis International School Park Lane Harbour（柏朗思观澜湖国际学校）
- University of California, Santa Cruz（加州大学圣克鲁兹分校）
- National Clinical Research Center of Geriatric Disorders, Xiangya Hospital, Central South University（中南大学湘雅医院国家老年疾病临床医学研究中心）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出视觉-语言基础模型BrainVLM，基于术前MRI等多模态数据，实现12种脑肿瘤的精确分类、不确定性量化及放射学报告生成，并通过大规模验证和临床研究证明其有效性。
AI中文摘要：
背景：基于磁共振成像（MRI）的脑肿瘤类型非侵入性术前诊断至关重要，但由于不同肿瘤类型之间的影像特征重叠、观察者间差异以及获得专业知识所需的大量培训，这一任务具有挑战性。我们旨在开发一种基于MRI的人工智能（AI）模型，用于自动、可靠的脑肿瘤分类，并具备诊断不确定性量化和放射学报告生成功能。方法：我们开发了BrainVLM，用于对世界卫生组织（WHO）2021年分类的所有12种脑肿瘤类型进行分类。BrainVLM整合了不确定性量化策略以指示预测可靠性，以及一个生成放射学报告以阐明临床依据的模块。BrainVLM在来自40,043名个体的多模态数据（MRI扫描、人口统计学信息和放射学报告）上进行了训练。并在5,211名经病理确诊的脑肿瘤患者上进行了验证，其中包括来自主要医院的3,877名保留患者和来自11家独立医院的1,334名患者。我们进一步开展了两项概念验证研究，以验证其在AI-临床医生工作流程中的临床实用性：1）一项盲法多读者研究，12名不同经验水平的神经放射科医生在有无AI辅助的情况下解读了248例回顾性病例；2）一项真实世界前瞻性研究，其中1,009名患者在术前由BrainVLM和放射科医生独立、盲法评估。此外，我们使用一个包含632名患者的多中心队列，证明了BrainVLM在成人型弥漫性胶质瘤术前分子亚型预测中的实用性。
英文摘要：
We developed BrainVLM to classify all 12 World Health Organization (WHO) 2021 brain tumor types. BrainVLM integrates an uncertainty quantification strategy to indicate prediction reliability and a module for generating radiology reports to elucidate the clinical rationale. BrainVLM was trained on multi-modal data (MRI scans, demographics, and radiology reports) from 40,043 individuals. It was validated on 5,211 patients with pathologically confirmed brain tumors, including 3,877 held-out patients from the primary hospital and 1,334 patients from 11 independent hospitals. We further conducted two proof-of-concept studies to validate its clinical utility in AI-clinician workflows: 1) a blinded multireader study where 12 neuroradiologists across varying experience levels interpreted 248 retrospective cases with or without AI assistance, and 2) a real-world prospective study in which 1,009 patients were independently and blindly assessed by BrainVLM and radiologists before surgery. Additionally, we demonstrated BrainVLM's utility in preoperative molecular subgroup prediction for adult-type diffuse gliomas, using a multi-center cohort of 632 patients. In primary evaluation, BrainVLM achieved an area under the curve (macro-AUC) of 0.85 (95% CI: 0.84-0.86), and an F1 score of 0.82 (95% CI: 0.81-0.83), surpassing neuroradiologists (F1 = 0.80 (95% CI: 0.79-0.81)). In external validation across 11 centers, BrainVLM achieved an AUC = 0.80 (95% CI: 0.79-0.82) and F1 = 0.75 (95% CI: 0.73-0.78), compared with F1 = 0.71 (95% CI: 0.69-0.73) for neuroradiologists. In prospective real-world evaluation, BrainVLM maintained performance comparable to neuroradiologists. The BrainVLM project page is available at https://hku-healthai.github.io/brainvlm_project.github.io/.
