为人工智能模型建立严谨且高性价比的临床试验
Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models
AI总结：
针对AI临床评估缺乏严谨且高性价比方法的问题，提出双中心AI随机对照试验（DC-AI RCTs）及虚拟临床医生模拟试验（VC-MedAI），基于14个中心125名医生的7500份记录验证了其有效性与必要性，为AI临床评估提供创新方法。
摘要：
医学领域的人工智能（AI）与临床实践之间仍存在巨大差距，主要原因是缺乏严谨且高性价比的评估方法。最先进和实际应用中的AI模型评估仅限于在医疗数据集上进行实验室研究，或者进行没有对照或仅以患者为中心对照的直接临床试验。此外，临床医生与AI协作的关键作用（这对决定AI对临床实践的影响至关重要）经常被忽视。我们首次强调了在临床实践中对AI模型采用严谨且高性价比评估方法的迫切必要性，提出以患者/临床医生为中心（双中心）的AI随机对照试验（DC-AI RCTs）以及基于虚拟临床医生的计算机模拟试验（VC-MedAI）作为DC-AI RCTs的有效替代。利用跨越14个医疗中心、涉及125名临床医生的两步初步DC-AI RCTs中的7500份诊断记录，我们的结果证明了DC-AI RCTs的必要性和VC-MedAI的有效性。值得注意的是，VC-MedAI的表现与人类临床医生相当，能够重现前瞻性DC-AI RCTs的见解和结论。我们期望DC-AI RCTs和VC-MedAI成为关键进步，为临床实践中的AI模型提供创新且变革性的评估方法，提供类似常规医学的临床前环境，并以高性价比和快速迭代的方式重塑开发范式。中国临床试验注册号：ChiCTR2400086816。
英文摘要：
A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of clinicians in collaborating with AI, pivotal for determining its impact on clinical practice, is often overlooked. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice, featuring patient/clinician-centered (dual-centered) AI randomized controlled trials (DC-AI RCTs) and virtual clinician-based in-silico trials (VC-MedAI) as an effective proxy for DC-AI RCTs. Leveraging 7500 diagnosis records from two-step inaugural DC-AI RCTs across 14 medical centers with 125 clinicians, our results demonstrate the necessity of DC-AI RCTs and the effectiveness of VC-MedAI. Notably, VC-MedAI performs comparably to human clinicians, replicating insights and conclusions from prospective DC-AI RCTs. We envision DC-AI RCTs and VC-MedAI as pivotal advancements, presenting innovative and transformative evaluation methodologies for AI models in clinical practice, offering a preclinical-like setting mirroring conventional medicine, and reshaping development paradigms in a cost-effective and fast-iterative manner. Chinese Clinical Trial Registration: ChiCTR2400086816.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
