利用大型语言模型增强临床笔记中的表型识别:PhenoBCBERT和PhenoGPT
Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT
AI总结:
本研究提出PhenoBCBERT和PhenoGPT两种模型,利用大型语言模型提升临床笔记中表型术语的自动识别与提取,从而推动疾病相关生物学研究。
摘要:
我们假设基于transformer架构的大型语言模型(LLMs)可以实现临床表型术语的自动检测,包括未在HPO中记录的术语。在本研究中,我们开发了两种模型:PhenoBCBERT是一种基于BERT的模型,利用Bio+Clinical BERT作为其预训练模型;PhenoGPT是一种基于GPT的模型,可以从多种GPT模型中初始化,包括开源版本如GPT-J、Falcon和LLaMA,以及闭源版本如GPT-3和GPT-3.5。我们比较了我们的方法与PhenoTagger,一种结合规则和深度学习方法的HPO识别工具。我们发现我们的方法可以提取更多表型概念,包括未由HPO表征的新术语。我们还对生物医学文献进行了案例研究,以说明如何识别和提取新的表型信息。我们还比较了当前基于BERT和基于GPT的模型在表型标记中的多个方面,包括模型架构、内存使用、速度、准确性和隐私保护。我们还讨论了在transformer模型中添加否定步骤和HPO规范化层以提高HPO术语标记。结论是,PhenoBCBERT和PhenoGPT能够从临床笔记和生物医学文献中自动发现表型术语,促进自动下游任务以获得人类疾病的新生物学见解。
英文摘要:
We hypothesize that large language models (LLMs) based on the transformer architecture can enable automated detection of clinical phenotype terms, including terms not documented in the HPO. In this study, we developed two types of models: PhenoBCBERT, a BERT-based model, utilizing Bio+Clinical BERT as its pre-trained model, and PhenoGPT, a GPT-based model that can be initialized from diverse GPT models, including open-source versions such as GPT-J, Falcon, and LLaMA, as well as closed-source versions such as GPT-3 and GPT-3.5. We compared our methods with PhenoTagger, a recently developed HPO recognition tool that combines rule-based and deep learning methods. We found that our methods can extract more phenotype concepts, including novel ones not characterized by HPO. We also performed case studies on biomedical literature to illustrate how new phenotype information can be recognized and extracted. We compared current BERT-based versus GPT-based models for phenotype tagging, in multiple aspects including model architecture, memory usage, speed, accuracy, and privacy protection. We also discussed the addition of a negation step and an HPO normalization layer to the transformer models for improved HPO term tagging. In conclusion, PhenoBCBERT and PhenoGPT enable the automated discovery of phenotype terms from clinical notes and biomedical literature, facilitating automated downstream tasks to derive new biological insights on human diseases.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。