CLIMB：面向大规模多模态临床基础模型的数据基础
CLIMB: Data Foundations for Large Scale Multimodal Clinical Foundation Models
AI总结：
针对现有临床AI基准模态与任务有限的问题，提出整合多模态数据的大规模临床基准CLIMB，验证了多任务预训练对性能和泛化能力的提升，为临床AI发展提供数据基础。
摘要：
临床人工智能的最新进展已在众多临床领域取得显著突破。然而，现有基准和模型主要局限于少量模态与任务，阻碍了可对患者健康与福祉进行整体评估的大规模多模态方法的发展。为填补这一空白，我们提出Clinical Large-Scale Integrative Multimodal Benchmark（CLIMB，临床大规模整合多模态基准），这是一个整合了影像、语言、时序和图等多种模态临床数据的综合性临床基准。CLIMB包含451万份患者样本，总数据量达19.01太字节，涵盖2D影像、3D视频、时间序列、图以及多模态数据。通过大量实证评估，我们证明多任务预训练可显著提升在研究不足领域的性能，与单任务学习相比，超声分析性能提升最高达29%，心电图（ECG）分析提升最高达23%。在CLIMB上预训练还有效提升了模型对新任务的泛化能力，并且当搭配适配任务的融合策略时，强大的单模态编码器性能可很好地转化为多模态性能。我们的研究结果为新型架构设计和预训练策略提供了基础，以推动临床人工智能研究。代码已在https://github.com/DDVD233/climb发布。
英文摘要：
Recent advances in clinical AI have enabled remarkable progress across many clinical domains. However, existing benchmarks and models are primarily limited to a small set of modalities and tasks, which hinders the development of large-scale multimodal methods that can make holistic assessments of patient health and well-being. To bridge this gap, we introduce Clinical Large-Scale Integrative Multimodal Benchmark (CLIMB), a comprehensive clinical benchmark unifying diverse clinical data across imaging, language, temporal, and graph modalities. CLIMB comprises 4.51 million patient samples totaling 19.01 terabytes distributed across 2D imaging, 3D video, time series, graphs, and multimodal data. Through extensive empirical evaluation, we demonstrate that multitask pretraining significantly improves performance on understudied domains, achieving up to 29% improvement in ultrasound and 23% in ECG analysis over single-task learning. Pretraining on CLIMB also effectively improves models' generalization capability to new tasks, and strong unimodal encoder performance translates well to multimodal performance when paired with task-appropriate fusion strategies. Our findings provide a foundation for new architecture designs and pretraining strategies to advance clinical AI research. Code is released at https://github.com/DDVD233/climb.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
