arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.00731cs.CLcs.AIcs.LG
收藏

大语言模型在词元级临床命名实体识别中表现不佳

Large Language Models Struggle in Token-Level Clinical Named Entity Recognition

McWilliams School of Biomedical Informatics, University of Texas Health Science Center(德克萨斯大学休斯顿健康科学中心麦克威廉姆斯生物医学信息学院)

Qiuhao Lu, Rui Li, Andrew Wen, Jinlian Wang, Liwei Wang, Hongfang Liu

AI总结:

本研究针对当前临床词元级命名实体识别(尤其是本地开源大语言模型应用)的研究空白,通过零样本/少样本提示、检索增强生成、指令微调等实验,探究了专有与本地大语言模型在该任务中的效果,揭示了其在罕见病场景下的固有挑战并提出改进方向,填补了医疗健康信息学领域的相关空白。

摘要:

大语言模型(LLMs)已革新了多个领域,包括医疗健康领域,它们在该领域有着多样的应用。在罕见病场景下,数据稀缺、复杂性和特殊性带来了巨大挑战,LLMs的效用尤为显著。在临床领域,命名实体识别(NER)是一项核心任务,在从临床文本中提取相关信息方面发挥着关键作用。尽管LLMs前景可观,但当前研究大多集中在文档级NER上,即在整个文档的更通用语境中识别实体,却不提取其精确位置。此外,已有研究致力于让ChatGPT适配词元级NER。然而,针对临床文本开展词元级NER的研究仍存在显著空白,尤其是在使用本地开源LLMs的情况下。本研究旨在通过探究专有LLMs和本地LLMs在词元级临床NER中的效果来填补这一空白。本质上,我们通过一系列涉及零样本提示、少样本提示、检索增强生成(RAG)和指令微调的实验,深入研究了这些模型的能力。我们的探索揭示了LLMs在词元级NER中面临的固有挑战(尤其是在罕见病场景下),并为其在医疗健康领域的应用提出了可能的改进方向。本研究有助于缩小医疗健康信息学领域的一项重要空白,并提供了可推动LLMs在医疗健康领域更精细化应用的见解。

英文摘要:

Large Language Models (LLMs) have revolutionized various sectors, including healthcare where they are employed in diverse applications. Their utility is particularly significant in the context of rare diseases, where data scarcity, complexity, and specificity pose considerable challenges. In the clinical domain, Named Entity Recognition (NER) stands out as an essential task and it plays a crucial role in extracting relevant information from clinical texts. Despite the promise of LLMs, current research mostly concentrates on document-level NER, identifying entities in a more general context across entire documents, without extracting their precise location. Additionally, efforts have been directed towards adapting ChatGPT for token-level NER. However, there is a significant research gap when it comes to employing token-level NER for clinical texts, especially with the use of local open-source LLMs. This study aims to bridge this gap by investigating the effectiveness of both proprietary and local LLMs in token-level clinical NER. Essentially, we delve into the capabilities of these models through a series of experiments involving zero-shot prompting, few-shot prompting, retrieval-augmented generation (RAG), and instruction-fine-tuning. Our exploration reveals the inherent challenges LLMs face in token-level NER, particularly in the context of rare diseases, and suggests possible improvements for their application in healthcare. This research contributes to narrowing a significant gap in healthcare informatics and offers insights that could lead to a more refined application of LLMs in the healthcare sector.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑