arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.14779cs.CL
收藏

Med42 -- 评估医疗大语言模型的微调策略:全参数与参数高效方法

Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches

G42 Healthcare(G42 医疗健康);Core42;Cerebras Systems
Clément Christophe, Praveen K Kanithi, Prateek Munjal, Tathagata Raha, Nasir Hayat, Ronnie Rajan, Ahmed Al-Mahrooqi, Avani Gupta, Muhammad Umar Salman, Gurpreet… 展开作者

Clément Christophe, Praveen K Kanithi, Prateek Munjal, Tathagata Raha, Nasir Hayat, Ronnie Rajan, Ahmed Al-Mahrooqi, Avani Gupta, Muhammad Umar Salman, Gurpreet Gosal, Bhargav Kanakiya, Charles Chen, Natalia Vassilieva, Boulbaba Ben Amor, Marco AF Pimentel, Shadab Khan

AI总结:

本研究比较了医疗大语言模型的全参数与参数高效微调策略,基于 Llama-2 架构开发了 Med42 模型,在 USMLE 数据集上达到 72% 准确率,为开源医疗 LLM 设定新标准。

摘要:

本研究对两种主要的微调方法——全参数微调和参数高效微调——在医疗大语言模型(LLMs)背景下进行了全面的分析与比较。我们基于 Llama-2 架构开发并优化了一系列 LLMs,专门用于增强医学知识检索、推理和问答能力。我们的实验系统评估了这些微调策略在多个知名医疗基准上的有效性。值得注意的是,我们的医疗 LLM Med42 在美国医师执照考试(USMLE)数据集上达到了 72% 的准确率,为开源医疗 LLMs 创立了新的性能标准。通过这项比较分析,我们旨在确定医疗领域微调 LLMs 最有效且高效的方法,从而为 AI 驱动的医疗应用的发展做出重大贡献。

英文摘要:

This study presents a comprehensive analysis and comparison of two predominant fine-tuning methodologies - full-parameter fine-tuning and parameter-efficient tuning - within the context of medical Large Language Models (LLMs). We developed and refined a series of LLMs, based on the Llama-2 architecture, specifically designed to enhance medical knowledge retrieval, reasoning, and question-answering capabilities. Our experiments systematically evaluate the effectiveness of these tuning strategies across various well-known medical benchmarks. Notably, our medical LLM Med42 showed an accuracy level of 72% on the US Medical Licensing Examination (USMLE) datasets, setting a new standard in performance for openly available medical LLMs. Through this comparative analysis, we aim to identify the most effective and efficient method for fine-tuning LLMs in the medical domain, thereby contributing significantly to the advancement of AI-driven healthcare applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑