arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.09019cs.CL
收藏

MedMobile: 一种具有临床能力的移动尺寸语言模型

MedMobile: A mobile-sized language model with clinical capabilities

NYU Langone Medical Center(纽约大学朗格尼医学中心);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Washington University School of Medicine in St. Louis(圣路易斯华盛顿大学医学院);NYU Langone Health(纽约大学朗格尼健康中心);New York University(纽约大学)

Krithik Vishwanath, Jaden Stryker, Anton Alyakin, Daniel Alexander Alber, Eric Karl Oermann

AI总结:

MedMobile是一种移动尺寸的语言模型,通过高效的方法在医疗领域实现高性能表现,成为参数最少且性能最佳的临床应用模型。

摘要:

语言模型(LMs)在医学领域已展示了专家级的推理和记忆能力。然而,计算成本和隐私问题正成为大规模实施的障碍。为了解决这些显著的限制,我们介绍了一种对phi-3-mini的简洁适应,即MedMobile,一个具有38亿参数的语言模型,能够运行在移动设备上,用于医疗应用。我们进行了一系列精心的管道添加,并展示了链式思维、集成和微调导致了最大的性能提升,而意外的检索增强生成未能展示显著的改进。我们在MultiMedQA和MedBullets上评估了我们管道的效率。我们证明MedMobile在MedQA(USMLE)上得分75.7%,超过了合格医生的通过线(约60%),并与比它大100倍的模型得分相当。在MultiMedQA的全部范围内,MedMobile实现了参数少于50亿的模型的最佳性能,并代表了通过MedQA(USMLE)的最小模型。MedMobile有潜力使语言模型在医学中的应用民主化,增强较低的计算需求和快速的推理速度。凭借能够克服语言模型在医学中最大的进入障碍的能力,我们希望MedMobile是开发具有临床相关性语言模型的关键一步。

英文摘要:

Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-scale implementation. To address these significant limitations, we introduce a parsimonious adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of running on a mobile device, for medical applications. We perform a careful set of pipeline additions and demonstrate that chain of thought, ensembling, and fine-tuning lead to the greatest performance gains, while unexpectedly retrieval augmented generation fails to demonstrate significant improvements. We evaluate the efficiency of our pipeline on the MultiMedQA and MedBullets. We demonstrate that MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for licensed physicians (~60%) and rivaling scores of models 100 times its size. Across the entirety of the MultiMedQA, MedMobile achieves SOTA performance for models with less than 5B parameters and represents the smallest model to pass the MedQA (USMLE). MedMobile holds promise to democratize access to language models in medicine, bolstering lower compute needs and fast inference speeds. With the ability to combat the biggest barriers to entry for language models in medicine, we hope that MedMobile is a critical step forward in developing clinically relevant language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑