RAMIE：基于大语言模型在膳食补充剂上的检索增强多任务信息提取
RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements
AI总结：
本文提出RAMIE框架，结合指令微调与RAG技术从临床记录中多任务提取膳食补充剂信息，实验证明其显著提升了Llama2等模型的F1分数与整体准确率。
摘要：
目标：我们旨在开发一个先进的多任务大语言模型（LLM）框架，以从临床记录中提取关于膳食补充剂（DS）的多类信息。方法：我们使用四个核心DS信息提取任务——即命名实体识别（NER：2,949个临床句子）、关系抽取（RE：4,892个句子）、三元组抽取（TE：2,949个句子）和用法分类（UC：2,460个句子）作为我们的多任务。我们引入了新颖的Retrieval-Augmented Multi-task Information Extraction (RAMIE) Framework，包括：1）采用带有任务特定提示的指令微调技术，2）训练LLMs以处理多个任务，提高存储效率并降低训练成本，3）通过从训练集中检索相似示例，结合检索增强生成（RAG）技术。我们将RAMIE的性能与仅使用指令微调的LLMs进行比较，并进行了消融研究，以评估多任务学习和RAG对提高多任务性能的贡献。结果：在RAMIE框架的帮助下，Llama2-13B在NER任务上取得了87.39的F1分数（提升3.51%），并在RE任务上表现出色，F1分数为93.74（提升1.15%）。在TE任务上，Llama2-7B得分为79.45（提升14.26%），而MedAlpaca-7B在UC任务上取得了93.45的最高F1分数（提升0.94%）。消融研究表明，虽然MTL以轻微的性能折衷提高了效率，但RAG显著提升了整体准确率。结论：本研究提出了一个新颖的RAMIE框架，在从临床记录中提取DS相关数据的多任务信息提取方面显示出显著改进。我们的框架有潜力应用于其他领域。
英文摘要：
\textbf{Objective:} We aimed to develop an advanced multi-task large language model (LLM) framework to extract multiple types of information about dietary supplements (DS) from clinical records. \textbf{Methods:} We used four core DS information extraction tasks - namely, named entity recognition (NER: 2,949 clinical sentences), relation extraction (RE: 4,892 sentences), triple extraction (TE: 2,949 sentences), and usage classification (UC: 2,460 sentences) as our multitasks. We introduced a novel Retrieval-Augmented Multi-task Information Extraction (RAMIE) Framework, including: 1) employed instruction fine-tuning techniques with task-specific prompts, 2) trained LLMs for multiple tasks with improved storage efficiency and lower training costs, and 3) incorporated retrieval augmentation generation (RAG) techniques by retrieving similar examples from the training set. We compared RAMIE's performance to LLMs with instruction fine-tuning alone and conducted an ablation study to assess the contributions of multi-task learning and RAG to improved multitasking performance. \textbf{Results:} With the aid of the RAMIE framework, Llama2-13B achieved an F1 score of 87.39 (3.51\% improvement) on the NER task and demonstrated outstanding performance on the RE task with an F1 score of 93.74 (1.15\% improvement). For the TE task, Llama2-7B scored 79.45 (14.26\% improvement), and MedAlpaca-7B achieved the highest F1 score of 93.45 (0.94\% improvement) on the UC task. The ablation study revealed that while MTL increased efficiency with a slight trade-off in performance, RAG significantly boosted overall accuracy. \textbf{Conclusion:} This study presents a novel RAMIE framework that demonstrates substantial improvements in multi-task information extraction for DS-related data from clinical records. Our framework can potentially be applied to other domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
