临床诊疗中的医学多模态基础模型：应用、挑战与未来方向
Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions
- School of Biomedical Engineering, Tsinghua University（清华大学生物医学工程学院）
- Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
- Hepato-Pancreato-Biliary Center, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University（清华大学临床医学院北京清华长庚医院肝胆胰中心）
- School of Computer and Communication Engineering, University of Science and Technology Beijing（北京科技大学计算机与通信工程学院）
- College of Electronic Information Engineering, Tongji University（同济大学电子与信息工程学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文综述医学多模态基础模型（MMFMs）在临床诊疗中的数据集、架构与应用进展，分析多模态表征优化的挑战和机遇，并展望其对精准医疗的推动作用。
AI中文摘要：
近期深度学习的进步显著革新了临床诊断与治疗领域，提供了在多种临床场景中提升诊断精度和治疗效果的新途径，从而推动了精准医学的发展。多器官、多模态数据集日益丰富，加速了大规模医学多模态基础模型（MMFMs）的开发。这些模型以强大的泛化能力和丰富的表征能力著称，正越来越多地被用于解决从早期诊断到个性化治疗策略的广泛临床任务。本综述全面分析了MMFMs的最新进展，聚焦数据集、模型架构和临床应用三个关键方面。我们还探讨了优化多模态表征所面临的挑战与机遇，并讨论这些进展如何通过改善患者结局和提高临床工作流程效率来塑造医疗保健的未来。
英文摘要：
Recent advancements in deep learning have significantly revolutionized the field of clinical diagnosis and treatment, offering novel approaches to improve diagnostic precision and treatment efficacy across diverse clinical domains, thus driving the pursuit of precision medicine. The growing availability of multi-organ and multimodal datasets has accelerated the development of large-scale Medical Multimodal Foundation Models (MMFMs). These models, known for their strong generalization capabilities and rich representational power, are increasingly being adapted to address a wide range of clinical tasks, from early diagnosis to personalized treatment strategies. This review offers a comprehensive analysis of recent developments in MMFMs, focusing on three key aspects: datasets, model architectures, and clinical applications. We also explore the challenges and opportunities in optimizing multimodal representations and discuss how these advancements are shaping the future of healthcare by enabling improved patient outcomes and more efficient clinical workflows.
