arXiv 2502.022492025-02-04cs.CLcs.AI

基于微调与检索增强生成的Medicare对话式AI

Conversation AI Dialog for Medicare powered by Finetuning and Retrieval Augmented Generation

University of Florida（佛罗里达大学）；National Institute of Technology Calicut（卡利卡特国家理工学院）；SRM, Chennai（SRM大学（金奈））；University of Cincinnati（辛辛那提大学）；Dartmouth College（达特茅斯学院）；Arizona State University（亚利桑那州立大学）

Atharva Mangeshkumar Agrawal, Rutika Pandurang Shinde, Vasanth Kumar Bhukya, Ashmita Chakraborty, Sagar Bharat Shah, Tanmay Shukla, Sree Pradeep Kumar Relangi, Nilesh Mutyam

备注：: 12 pages
发表信息：: ResMilitaris 2022

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究比较了LoRA微调与RAG框架在医患对话中的表现，评估Llama-2、GPT和LSTM模型的语言质量、事实准确性及安全性，并探索领域知识整合对LLM性能的提升作用。

摘要：

大型语言模型（LLMs）在自然语言处理任务（包括对话生成）中展现出令人印象深刻的能力。本研究旨在对两种突出技术——使用LoRA（Low-Rank Adaptation，低秩适配）的微调与检索增强生成（Retrieval-Augmented Generation，RAG）框架——在混合医学领域的多数据集医患聊天对话背景下进行新颖的比较分析。该分析涉及三个最先进的模型：Llama-2、GPT和LSTM模型。利用真实的医患对话，我们全面评估模型性能，评估关键指标，如语言质量（困惑度、BLEU分数）、事实准确性（对照医学知识库进行事实核查）、对医学指南的遵守情况以及整体人类判断（连贯性、同理心、安全性）。研究结果为每种方法的优势和局限性提供了见解，揭示了它们在医疗保健应用中的适用性。此外，本研究调查了模型在处理多样化患者查询（从一般健康咨询到特定医疗状况）方面的鲁棒性。还探讨了领域特定知识整合的影响，突出显示了通过有针对性的数据增强和检索策略提升LLM性能的潜力。

英文摘要：

Large language models (LLMs) have shown impressive capabilities in natural language processing tasks, including dialogue generation. This research aims to conduct a novel comparative analysis of two prominent techniques, fine-tuning with LoRA (Low-Rank Adaptation) and the Retrieval-Augmented Generation (RAG) framework, in the context of doctor-patient chat conversations with multiple datasets of mixed medical domains. The analysis involves three state-of-the-art models: Llama-2, GPT, and the LSTM model. Employing real-world doctor-patient dialogues, we comprehensively evaluate the performance of models, assessing key metrics such as language quality (perplexity, BLEU score), factual accuracy (fact-checking against medical knowledge bases), adherence to medical guidelines, and overall human judgments (coherence, empathy, safety). The findings provide insights into the strengths and limitations of each approach, shedding light on their suitability for healthcare applications. Furthermore, the research investigates the robustness of the models in handling diverse patient queries, ranging from general health inquiries to specific medical conditions. The impact of domain-specific knowledge integration is also explored, highlighting the potential for enhancing LLM performance through targeted data augmentation and retrieval strategies.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。