发表机构
Mashhad University of Medical Sciences; Northside Hospital; Mayo Clinic; University of Pennsylvania（马什哈德医科大学; 北院医院; 梅奥诊所; 宾夕法尼亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本教程提供医学影像不确定性量化的19节实践课程，并引入LLM评估框架，验证其能提升模型知识检索准确率与AUROC。
AI 中文摘要
不确定性量化（UQ）日益被认为是医学影像中可靠机器学习的重要组成部分，然而连接UQ理论、实现和评估的实用资源仍然有限。我们开发了医学影像分析中的不确定性量化（UQMIA），这是一个开放获取的实践教程，包含19个部分，涵盖了主要的UQ方法，包括变分推断、蒙特卡洛dropout、深度集成、证据深度学习以及共形预测，同时还包括评估不确定性可靠性的方法。该教程面向具有医学影像经验的研究人员和从业者，从基础概念逐步深入到实现和评估，并提供了可通过Kaggle执行的笔记本。除了介绍教程外，我们引入了一个使用大语言模型（LLM）的框架，用于评估技术教育资源是否包含可检索和可用的知识。使用源自主要方法学文献的100道四选一选择题，我们评估了来自六个模型家族的20个指令调优LLM，分别在有无检索教程上下文的情况下进行。检索UQMIA提高了20个模型中18个的准确率，平均准确率从0.680提升到0.742（+0.062；Holm校正P=0.00032），并改善了20个模型中18个的受试者工作特征曲线下面积（AUROC），从0.725提升到0.788（+0.063；Holm校正P=0.00032）。UQMIA为学习医学影像中的UQ提供了易获取的资源，并展示了一种基于LLM的评估技术教育材料的方法。该教程可在https URL获取，其源代码和笔记本可在https URL获取。
英文摘要
Uncertainty quantification (UQ) is increasingly recognized as an important component of reliable machine learning in medical imaging, yet practical resources connecting UQ theory, implementation, and evaluation remain limited. We developed Uncertainty Quantification in Medical Imaging Analysis (UQMIA), an open-access, hands-on tutorial comprising 19 sessions covering major UQ approaches, including variational inference, Monte Carlo dropout, deep ensembles, evidential deep learning, and conformal prediction, together with methods for evaluating uncertainty reliability. The tutorial is designed for researchers and practitioners with experience in medical imaging, progressing from foundational concepts to implementation and evaluation, with notebooks executable through Kaggle. Beyond presenting the tutorial, we introduce a framework using large language models (LLMs) to evaluate whether technical educational resources contain retrievable and usable knowledge. Using 100 four-option multiple-choice questions derived from primary methodological literature, we evaluated 20 instruction-tuned LLMs from six model families with and without retrieved tutorial context. Retrieval of UQMIA improved accuracy in 18 of 20 models, increasing mean accuracy from 0.680 to 0.742 (+0.062; Holm-adjusted P=0.00032), and improved area under the receiver operating characteristic curve (AUROC) in 18 of 20 models, increasing from 0.725 to 0.788 (+0.063; Holm-adjusted P=0.00032). UQMIA provides an accessible resource for learning UQ in medical imaging and demonstrates an LLM-based approach for evaluating technical educational material. The tutorial is available at https://benyamin-gheiji.github.io/Uncertainty-Quantification-Medical-Imaging-Analysis/ and its source code and notebooks at https://github.com/benyamin-gheiji/Uncertainty-Quantification-Medical-Imaging-Analysis/.
