大型语言模型在零样本临床自然语言处理中提示策略的实证评估
An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing
AI总结：
本文对GPT-3.5、BARD和LLAMA2在五个零样本临床NLP任务上的提示工程进行了系统评估，提出了启发式和集成提示，并对比零样本与少样本提示，为临床NLP的提示设计提供了新见解。
摘要：
大型语言模型（LLMs）在自然语言处理（NLP）中展现出了卓越的能力，尤其是在标注数据稀缺或昂贵的领域，如临床领域。然而，为了解锁这些LLMs中隐藏的临床知识，我们需要设计有效的提示，以引导它们在没有任何任务特定训练数据的情况下执行特定的临床NLP任务。这被称为上下文学习，它是一门艺术和科学，需要理解不同LLMs和提示工程方法的优势与劣势。在本文中，我们对五个临床NLP任务的提示工程进行了全面且系统的实验研究：临床词义消歧、生物医学证据抽取、共指消解、药物状态抽取和药物属性抽取。我们评估了近期文献中提出的提示，包括简单前缀、简单完形填空、思维链和预期提示，并引入了两种新类型的提示，即启发式提示和集成提示。我们在三个最先进的LLMs上评估了这些提示的性能：GPT-3.5、BARD和LLAMA2。我们还对比了零样本提示与少样本提示，并为临床NLP中LLMs的提示工程提供了新颖的见解和指南。据我们所知，这是生成式AI时代最早对临床NLP不同提示工程方法进行实证评估的工作之一，我们希望它能启发并为该领域的未来研究提供参考。
英文摘要：
Large language models (LLMs) have shown remarkable capabilities in Natural Language Processing (NLP), especially in domains where labeled data is scarce or expensive, such as clinical domain. However, to unlock the clinical knowledge hidden in these LLMs, we need to design effective prompts that can guide them to perform specific clinical NLP tasks without any task-specific training data. This is known as in-context learning, which is an art and science that requires understanding the strengths and weaknesses of different LLMs and prompt engineering approaches. In this paper, we present a comprehensive and systematic experimental study on prompt engineering for five clinical NLP tasks: Clinical Sense Disambiguation, Biomedical Evidence Extraction, Coreference Resolution, Medication Status Extraction, and Medication Attribute Extraction. We assessed the prompts proposed in recent literature, including simple prefix, simple cloze, chain of thought, and anticipatory prompts, and introduced two new types of prompts, namely heuristic prompting and ensemble prompting. We evaluated the performance of these prompts on three state-of-the-art LLMs: GPT-3.5, BARD, and LLAMA2. We also contrasted zero-shot prompting with few-shot prompting, and provide novel insights and guidelines for prompt engineering for LLMs in clinical NLP. To the best of our knowledge, this is one of the first works on the empirical evaluation of different prompt engineering approaches for clinical NLP in this era of generative AI, and we hope that it will inspire and inform future research in this area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
