InfiMed-Foundation：以计算高效预训练与多阶段微调开创先进多模态医学模型
InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuning
AI总结：
本文提出医学专用多模态大模型 InfiMed-Foundation-1.7B 和 4B，通过五维质量评估、低到高分辨率训练、序列打包及三阶段微调，在 MedEvalKit 上超越多个基线模型。
摘要：
多模态大语言模型（MLLMs）已在多个领域展现出显著潜力，但其在医学领域的应用仍受到若干挑战的阻碍。通用型多模态大语言模型往往缺乏医学任务所需的专业知识，导致回答不确定或产生幻觉。从先进模型进行知识蒸馏难以捕获放射学和药理学中的领域专长。此外，利用大规模医学数据进行持续预训练所带来的计算成本构成了显著的效率挑战。为解决这些问题，我们提出 InfiMed-Foundation-1.7B 和 InfiMed-Foundation-4B，这是两个医学专用多模态大语言模型，旨在医学应用中提供最先进的性能。我们结合了高质量通用与医学多模态数据，并提出一种新颖的五维质量评估框架，以构建高质量多模态医学数据集。我们采用从低到高的图像分辨率和多模态序列打包来提升训练效率，从而能够整合大规模医学数据。此外，三阶段监督微调流程确保为复杂医学任务有效提取知识。在 MedEvalKit 框架上评估，InfiMed-Foundation-1.7B 优于 Qwen2.5VL-3B，而 InfiMed-Foundation-4B 超越 HuatuoGPT-V-7B 和 MedGemma-27B-IT，在医学视觉问答和诊断任务中展现出更优性能。通过应对数据质量、训练效率和领域特定知识提取方面的关键挑战，我们的工作为医疗保健领域更可靠、更有效的 AI 驱动解决方案铺平了道路。InfiMed-Foundation-4B 模型可在 https://huggingface.co/InfiX-ai/InfiMed-Foundation-4B 获取。
英文摘要：
Multimodal large language models (MLLMs) have shown remarkable potential in various domains, yet their application in the medical field is hindered by several challenges. General-purpose MLLMs often lack the specialized knowledge required for medical tasks, leading to uncertain or hallucinatory responses. Knowledge distillation from advanced models struggles to capture domain-specific expertise in radiology and pharmacology. Additionally, the computational cost of continual pretraining with large-scale medical data poses significant efficiency challenges. To address these issues, we propose InfiMed-Foundation-1.7B and InfiMed-Foundation-4B, two medical-specific MLLMs designed to deliver state-of-the-art performance in medical applications. We combined high-quality general-purpose and medical multimodal data and proposed a novel five-dimensional quality assessment framework to curate high-quality multimodal medical datasets. We employ low-to-high image resolution and multimodal sequence packing to enhance training efficiency, enabling the integration of extensive medical data. Furthermore, a three-stage supervised fine-tuning process ensures effective knowledge extraction for complex medical tasks. Evaluated on the MedEvalKit framework, InfiMed-Foundation-1.7B outperforms Qwen2.5VL-3B, while InfiMed-Foundation-4B surpasses HuatuoGPT-V-7B and MedGemma-27B-IT, demonstrating superior performance in medical visual question answering and diagnostic tasks. By addressing key challenges in data quality, training efficiency, and domain-specific knowledge extraction, our work paves the way for more reliable and effective AI-driven solutions in healthcare. InfiMed-Foundation-4B model is available at \href{https://huggingface.co/InfiX-ai/InfiMed-Foundation-4B}{InfiMed-Foundation-4B}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
