虚拟与现实：使用XCAT分类器对胸部计算机断层扫描的COVID-19分类器进行外部验证
Virtual vs. Reality: External Validation of COVID-19 Classifiers using XCAT Phantoms for Chest Computed Tomography
摘要：
人工智能模型在医学成像领域的研究一直受到泛化能力差的阻碍。在过去的一年里，随着深度学习在COVID-19诊断中的大量应用，这个问题尤其令人担忧。虚拟成像试验（VITs）可以为这些模型的客观评价提供解决方案。在这项利用VITs的工作中，我们创建了CVIT-COVID数据集，包括来自模拟COVID-19和不同COVID-19形态和成像特性下的正常体模模型的180个虚拟成像计算机断层扫描（CT）图像。我们评估了来自滑铁卢大学的开源深度学习模型的性能，该模型使用多机构数据进行训练，并使用名为MosMed的开放临床数据集进行训练。我们进一步验证了该模型对305个CT图像的开放临床数据的性能，以了解虚拟与真实临床数据的性能。该开源模型在原始滑铁卢数据集上的表现近乎完美，但在另一个临床数据集（AUC=0.77）和我们模拟的CVIT-COVID数据集（AUC=0.55）的外部测试中表现出一致的性能下降。在内部测试集（MosMed测试集）上进行测试时，内部模型的AUC为0.87。然而，当在临床和我们模拟的CVIT-COVID数据集上进行评估时，性能下降到0.65和0.69的AUC。VIT框架提供了对成像条件的控制，使我们能够证明，当CT曝光从28.5 mAs变为57 mAs时，性能没有变化。VIT框架还提供了体素水平的基础事实，揭示了对于弥漫性COVID-19感染大小>2.65%的肺体积，内部模型的性能在AUC=0.87时比对于体积<2.65%的局灶性疾病的AUC=0.52时高得多。虚拟成像框架实现了这些对模型性能的独特严格分析。
英文摘要：
Research studies of artificial intelligence models in medical imaging have been hampered by poor generalization. This problem has been especially concerning over the last year with numerous applications of deep learning for COVID-19 diagnosis. Virtual imaging trials (VITs) could provide a solution for objective evaluation of these models. In this work utilizing the VITs, we created the CVIT-COVID dataset including 180 virtually imaged computed tomography (CT) images from simulated COVID-19 and normal phantom models under different COVID-19 morphology and imaging properties. We evaluated the performance of an open-source, deep-learning model from the University of Waterloo trained with multi-institutional data and an in-house model trained with the open clinical dataset called MosMed. We further validated the model's performance against open clinical data of 305 CT images to understand virtual vs. real clinical data performance. The open-source model was published with nearly perfect performance on the original Waterloo dataset but showed a consistent performance drop in external testing on another clinical dataset (AUC=0.77) and our simulated CVIT-COVID dataset (AUC=0.55). The in-house model achieved an AUC of 0.87 while testing on the internal test set (MosMed test set). However, performance dropped to an AUC of 0.65 and 0.69 when evaluated on clinical and our simulated CVIT-COVID dataset. The VIT framework offered control over imaging conditions, allowing us to show there was no change in performance as CT exposure was changed from 28.5 to 57 mAs. The VIT framework also provided voxel-level ground truth, revealing that performance of in-house model was much higher at AUC=0.87 for diffuse COVID-19 infection size >2.65% lung volume versus AUC=0.52 for focal disease with <2.65% volume. The virtual imaging framework enabled these uniquely rigorous analyses of model performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
