提示大型语言模型用于临床时间关系抽取
Prompting Large Language Models for Clinical Temporal Relation Extraction
AI总结：
本文在2012 i2b2 CTRE任务上评估多种LLM的FFT与PEFT提示微调策略，全监督下GatorTron-Base硬提示以89.54% F1刷新SOTA，并分析了少样本设置差异。
摘要：
目的：本文旨在在少样本和全监督设置下，通过提示大型语言模型（LLM）进行临床时间关系抽取（CTRE）。材料与方法：本研究使用四种LLM：基于编码器的GatorTron-Base（345M）/Large（8.9B）；基于解码器的LLaMA3-8B/MeLLaMA-13B。我们开发了全量微调（FFT）和参数高效微调（PEFT）策略，并在2012 i2b2 CTRE任务上评估这些策略。我们为GatorTron-Base探索了四种微调策略：（1）标准微调，（2）使用未冻结LLM的硬提示，（3）使用冻结LLM的软提示，以及（4）使用冻结LLM的低秩适配（LoRA）。对于GatorTron-Large，我们利用量化技术评估了两种PEFT策略——软提示和使用冻结LLM的LoRA。此外，LLaMA3-8B和MeLLaMA-13B采用两种PEFT策略：通过指令微调和标准微调应用于冻结LLM的带量化LoRA（QLoRA）策略。结果：在全监督设置下，使用未冻结GatorTron-Base的硬提示取得最高F1分数（89.54%），超过SOTA模型（85.70%）3.74%。此外，适配GatorTron-Large的两种QLoRA变体以及GatorTron-Base的标准微调分别超过SOTA模型2.36%、1.88%和0.25%。在此设置下，具有冻结参数的基于解码器模型优于其基于编码器的对应模型；然而，该趋势在少样本场景中发生逆转。讨论与结论：本研究提出了显著提升CTRE性能的新方法，使依赖CTRE系统的下游任务受益。研究结果强调了根据任务需求和数据可用性选择合适模型及微调策略的重要性。未来工作将探索更大的模型和更广泛的CTRE应用。
英文摘要：
Objective: This paper aims to prompt large language models (LLMs) for clinical temporal relation extraction (CTRE) in both few-shot and fully supervised settings. Materials and Methods: This study utilizes four LLMs: Encoder-based GatorTron-Base (345M)/Large (8.9B); Decoder-based LLaMA3-8B/MeLLaMA-13B. We developed full (FFT) and parameter-efficient (PEFT) fine-tuning strategies and evaluated these strategies on the 2012 i2b2 CTRE task. We explored four fine-tuning strategies for GatorTron-Base: (1) Standard Fine-Tuning, (2) Hard-Prompting with Unfrozen LLMs, (3) Soft-Prompting with Frozen LLMs, and (4) Low-Rank Adaptation (LoRA) with Frozen LLMs. For GatorTron-Large, we assessed two PEFT strategies-Soft-Prompting and LoRA with Frozen LLMs-leveraging Quantization techniques. Additionally, LLaMA3-8B and MeLLaMA-13B employed two PEFT strategies: LoRA strategy with Quantization (QLoRA) applied to Frozen LLMs using instruction tuning and standard fine-tuning. Results: Under fully supervised settings, Hard-Prompting with Unfrozen GatorTron-Base achieved the highest F1 score (89.54%), surpassing the SOTA model (85.70%) by 3.74%. Additionally, two variants of QLoRA adapted to GatorTron-Large and Standard Fine-Tuning of GatorTron-Base exceeded the SOTA model by 2.36%, 1.88%, and 0.25%, respectively. Decoder-based models with frozen parameters outperformed their Encoder-based counterparts in this setting; however, the trend reversed in few-shot scenarios. Discussions and Conclusions: This study presented new methods that significantly improved CTRE performance, benefiting downstream tasks reliant on CTRE systems. The findings underscore the importance of selecting appropriate models and fine-tuning strategies based on task requirements and data availability. Future work will explore larger models and broader CTRE applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
