SA-MDKIF：面向大语言模型的可扩展可适配医学领域知识注入框架
SA-MDKIF: A Scalable and Adaptable Medical Domain Knowledge Injection Framework for Large Language Models
AI总结：
针对大语言模型缺乏医学领域知识的问题，提出含技能训练与技能适配两阶段的SA-MDKIF框架，通过指令微调注入医学知识，在9项医学任务上性能提升10-20%，未见过任务最高提升30%。
摘要：
大语言模型（LLMs）的最新进展已在各类自然语言处理（NLP）任务中展现出卓越性能。然而，由于缺乏医学领域知识，其在医学领域的有效应用受到阻碍。本研究提出SA-MDKIF——一种可扩展且可适配的框架，旨在通过指令微调将医学知识注入通用大语言模型，从而实现对各类下游任务的适配能力。SA-MDKIF包含两个阶段：技能训练与技能适配。第一阶段，我们定义了12项基础医学技能，并基于构建的统一格式指令数据集，采用AdaLoRA训练这些技能。第二阶段，我们使用任务特定的下游数据训练技能路由（skill router），并在推理阶段通过该路由将习得的技能与大语言模型整合。在9项不同医学任务上的实验结果表明，SA-MDKIF相比原始大语言模型性能提升10-20%。值得注意的是，这种提升在未见过的医学任务上尤为显著，最高可达30%。
英文摘要：
Recent advances in large language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, their effective application in the medical domain is hampered by a lack of medical domain knowledge. In this study, we present SA-MDKIF, a scalable and adaptable framework that aims to inject medical knowledge into general-purpose LLMs through instruction tuning, thereby enabling adaptability for various downstream tasks. SA-MDKIF consists of two stages: skill training and skill adaptation. In the first stage, we define 12 basic medical skills and use AdaLoRA to train these skills based on uniformly formatted instructional datasets that we have constructed. In the next stage, we train the skill router using task-specific downstream data and use this router to integrate the acquired skills with LLMs during inference. Experimental results on 9 different medical tasks show that SA-MDKIF improves performance by 10-20% compared to the original LLMs. Notably, this improvement is particularly pronounced for unseen medical tasks, showing an improvement of up to 30%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
