arXiv 2410.189552025-03-09cs.CL

BioMistral-NLU：通过指令微调实现更具泛化能力的医学语言理解

BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuning

University of Washington（华盛顿大学）；George Mason University（乔治梅森大学）

Yujuan Velvin Fu, Giridhar Kaushik Ramachandran, Namu Park, Kevin Lybarger, Fei Xia, Ozlem Uzuner, Meliha Yetisgen

备注：: 3 figures an 5 tables; Accepted by AMIA 2025 Informatics Summit

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出医学NLU模型BioMistral-NLU，通过统一提示格式和多样化指令微调数据集MNLU-Instruct，在零样本医学NLU任务上超越BioMistral、ChatGPT和GPT-4。

摘要：

诸如ChatGPT这样的大语言模型（LLMs）在大规模且多样化的指令遵循语料上进行了微调，能够泛化到新任务。然而，这些经过指令微调的LLMs在需要领域知识、细粒度文本理解以及结构化数据抽取的专门医学自然语言理解（NLU）任务中往往表现不佳。为弥合这一差距，我们：（1）为7项重要的NLU任务提出了一种统一的提示格式，（2）利用多样化的现有开源医学NLU语料构建了一个指令微调数据集MNLU-Instruct，以及（3）通过在MNLU-Instruct上微调BioMistral，开发了BioMistral-NLU，一个可泛化的医学NLU模型。我们在零样本设定下，基于两个广泛采用的医学NLU基准BLUE和BLURB中的6项重要NLU任务对BioMistral-NLU进行评估。实验表明，我们的BioMistral-NLU优于原始BioMistral以及专有LLMs——ChatGPT和GPT-4。我们的数据集无关提示策略以及在多样化NLU任务上的指令微调步骤增强了LLMs在多样化医学NLU任务上的泛化能力。我们的消融实验表明，即使在训练实例总数保持不变的情况下，在更广泛的任务种类上进行指令微调也能提升下游零样本泛化能力。

英文摘要：

Large language models (LLMs) such as ChatGPT are fine-tuned on large and diverse instruction-following corpora, and can generalize to new tasks. However, those instruction-tuned LLMs often perform poorly in specialized medical natural language understanding (NLU) tasks that require domain knowledge, granular text comprehension, and structured data extraction. To bridge the gap, we: (1) propose a unified prompting format for 7 important NLU tasks, (2) curate an instruction-tuning dataset, MNLU-Instruct, utilizing diverse existing open-source medical NLU corpora, and (3) develop BioMistral-NLU, a generalizable medical NLU model, through fine-tuning BioMistral on MNLU-Instruct. We evaluate BioMistral-NLU in a zero-shot setting, across 6 important NLU tasks, from two widely adopted medical NLU benchmarks: BLUE and BLURB. Our experiments show that our BioMistral-NLU outperforms the original BioMistral, as well as the proprietary LLMs - ChatGPT and GPT-4. Our dataset-agnostic prompting strategy and instruction tuning step over diverse NLU tasks enhance LLMs' generalizability across diverse medical NLU tasks. Our ablation experiments show that instruction-tuning on a wider variety of tasks, even when the total number of training instances remains constant, enhances downstream zero-shot generalization.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。