面向专科的通用医疗AI用于胸部CT筛查
Specialty-Oriented Generalist Medical AI for Chest CT Screening
AI总结：
提出首个医疗多模态多任务基础模型M3FM，通过多模态问答框架整合胸部CT等49种临床数据，在17项肺癌筛查任务中优于单模态模型，实现专科化通用医疗AI。
摘要：
现代医疗记录包含大量多模态自由文本临床数据和来自放射学、心脏病学和数字病理学的影像数据。充分挖掘此类大数据需要多任务处理；否则，隐匿但重要的方面可能被忽略，对临床管理和人群医疗产生不利影响。尽管AI在单模态数据的单一任务中取得了显著成功，但由于数据整理和模型架构的双重挑战，开发通用医疗AI以结合多模态数据进行多任务的进展仍然相对缓慢。数据挑战涉及在个体患者层面查询和整理多模态结构化和非结构化文本、字母数字，特别是3D断层扫描，以用于实时决策，并在大规模上估计人群健康统计。模型挑战要求可扩展且适应性强的网络架构，以整合多模态数据集用于多种临床任务。在此，我们提出首创的医疗多模态多任务基础模型（M3FM），应用于肺癌筛查及相关任务。在我们整理了一个包含49种临床数据类型（包括163,725个胸部CT序列）和17个涉及LCS的医疗任务的全面多模态多任务数据集后，我们开发了一个多模态问答框架，作为统一的训练和推理策略，通过自由文本提示协同多模态信息并执行多项任务。M3FM持续优于最先进的单模态任务特定模型，识别对临床任务具有信息价值的多模态数据元素，并灵活适应具有小规模分布外数据集的新任务。作为面向专科的通用医疗AI模型，M3FM为医学其他领域的类似突破铺平了道路，缩小了专科医生与全科医生之间的差距。
英文摘要：
Modern medical records include a vast amount of multimodal free text clinical data and imaging data from radiology, cardiology, and digital pathology. Fully mining such big data requires multitasking; otherwise, occult but important aspects may be overlooked, adversely affecting clinical management and population healthcare. Despite remarkable successes of AI in individual tasks with single-modal data, the progress in developing generalist medical AI remains relatively slow to combine multimodal data for multitasks because of the dual challenges of data curation and model architecture. The data challenge involves querying and curating multimodal structured and unstructured text, alphanumeric, and especially 3D tomographic scans on an individual patient level for real-time decisions and on a scale to estimate population health statistics. The model challenge demands a scalable and adaptable network architecture to integrate multimodal datasets for diverse clinical tasks. Here we propose the first-of-its-kind medical multimodal-multitask foundation model (M3FM) with application in lung cancer screening and related tasks. After we curated a comprehensive multimodal multitask dataset consisting of 49 clinical data types including 163,725 chest CT series and 17 medical tasks involved in LCS, we develop a multimodal question-answering framework as a unified training and inference strategy to synergize multimodal information and perform multiple tasks via free-text prompting. M3FM consistently outperforms the state-of-the-art single-modal task-specific models, identifies multimodal data elements informative for clinical tasks and flexibly adapts to new tasks with a small out-of-distribution dataset. As a specialty-oriented generalist medical AI model, M3FM paves the way for similar breakthroughs in other areas of medicine, closing the gap between specialists and the generalist.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
