arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.10459cs.CL
收藏

CancerLLM:癌症领域的大型语言模型

CancerLLM: A Large Language Model in Cancer Domain

University of Minnesota Twin Cities(明尼苏达大学双城分校);Johns Hopkins University(约翰斯·霍普金斯大学);UTHealth Houston(休斯顿健康科学中心);Yale School of Medicine(耶鲁医学院)

Mingchen Li, Jiatan Huang, Jeremy Yeung, Anne Blaes, Steven Johnson, Hongfang Liu, Hua Xu, Rui Zhang

AI总结:

针对癌症领域缺乏专用LLM及现有模型计算成本高的问题,提出70亿参数的CancerLLM,经大规模临床与病理数据预训练及微调,在表型提取和诊断生成任务上达到SOTA,兼具高效性与鲁棒性。

摘要:

医学大型语言模型(LLMs)在多种医学自然语言处理任务上表现出色;然而,目前仍缺乏专门用于癌症领域表型识别和诊断的LLM。此外,这些LLM通常具有数十亿参数,使得医疗系统在计算上成本高昂。因此,在本研究中,我们提出了CancerLLM,这是一个拥有70亿参数、采用Mistral风格架构的模型,它在近270万份临床笔记和涵盖17种癌症类型的51.5万份病理报告上进行了预训练,随后在两个与癌症相关的任务上进行了微调,包括癌症表型提取和癌症诊断生成。我们的评估表明,CancerLLM取得了最先进的结果,表型提取的F1分数为91.78%,诊断生成的F1分数为86.81%。它优于现有的LLM,F1分数平均提高了9.23%。此外,与其他LLM相比,CancerLLM在时间和GPU使用上展现了其效率及鲁棒性。我们证明,CancerLLM有可能为推进癌症领域的临床研究和实践提供一个有效且稳健的解决方案。

英文摘要:

Medical Large Language Models (LLMs) have demonstrated impressive performance on a wide variety of medical NLP tasks; however, there still lacks a LLM specifically designed for phenotyping identification and diagnosis in cancer domain. Moreover, these LLMs typically have several billions of parameters, making them computationally expensive for healthcare systems. Thus, in this study, we propose CancerLLM, a model with 7 billion parameters and a Mistral-style architecture, pre-trained on nearly 2.7M clinical notes and over 515K pathology reports covering 17 cancer types, followed by fine-tuning on two cancer-relevant tasks, including cancer phenotypes extraction and cancer diagnosis generation. Our evaluation demonstrated that the CancerLLM achieves state-of-the-art results with F1 score of 91.78% on phenotyping extraction and 86.81% on disganois generation. It outperformed existing LLMs, with an average F1 score improvement of 9.23%. Additionally, the CancerLLM demonstrated its efficiency on time and GPU usage, and robustness comparing with other LLMs. We demonstrated that CancerLLM can potentially provide an effective and robust solution to advance clinical research and practice in cancer domain

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑