通过两阶段指令微调方法实现医学多语言大语言模型的民主化
Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach
AI总结：
本文构建 MMed-IFT 与 MMed-IFT-MC 多语言医学指令数据集，提出先注入通用医学知识、再微调多选题的两阶段方法，在英语及多语言基准上实现效率与性能兼顾。
摘要：
开源的多语言医学大语言模型（LLM）有潜力服务于不同地区语言多样化的人群。将通用 LLM 适配到医疗保健领域通常需要持续预训练，但这种方法计算成本高昂，有时并不现实。由于缺乏模型在多样化场景中进行有效理解和推理所需的更广泛领域知识，针对特定任务进行指令微调并不总能保证最佳性能。为应对这些挑战，我们推出了 MMed-IFT 和 MMed-IFT-MC 两个多语言指令微调数据集，包含六种语言的 20 多万条高质量医学样本。我们提出一种两阶段训练范式：第一阶段利用 MMed-IFT 注入通用医学知识，第二阶段利用 MMed-IFT-MC 对特定任务的多项选择题进行微调。我们的方法在英语和多语言基准测试上均取得了具有竞争力的结果，在计算效率与性能之间实现了平衡。我们计划未来在 https://github.com/SpassMed/Med-Llama3 公开数据集和模型权重。
英文摘要：
Open-source, multilingual medical large language models (LLMs) have the potential to serve linguistically diverse populations across different regions. Adapting generic LLMs for healthcare often requires continual pretraining, but this approach is computationally expensive and sometimes impractical. Instruction fine-tuning on a specific task may not always guarantee optimal performance due to the lack of broader domain knowledge that the model needs to understand and reason effectively in diverse scenarios. To address these challenges, we introduce two multilingual instruction fine-tuning datasets, MMed-IFT and MMed-IFT-MC, containing over 200k high-quality medical samples in six languages. We propose a two-stage training paradigm: the first stage injects general medical knowledge using MMed-IFT, while the second stage fine-tunes task-specific multiple-choice questions with MMed-IFT-MC. Our method achieves competitive results on both English and multilingual benchmarks, striking a balance between computational efficiency and performance. We plan to make our dataset and model weights public at \url{https://github.com/SpassMed/Med-Llama3} in the future.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
