arXiv 2403.097272024-03-12cs.CLcs.AIcs.LG

探究检索增强生成与微调在开发AI驱动知识型系统中的性能

Investigating the performance of Retrieval-Augmented Generation and fine-tuning for the development of AI-driven knowledge-based systems

Department of Data Science and Visualization, Faculty of Informatics, University of Debrecen（数据科学与可视化系，信息学院，德布勒恩大学）；Doctoral School of Informatics, University of Debrecen（信息学博士学校，德布勒恩大学）；Neumann Technology Platform, Neumann Nonprofit Ltd.（内梅尔技术平台，内梅尔非营利有限公司）；Data-Driven Health Division of National Laboratory for Health Security, Health Services Management Training Centre, Semmelweis University（国家安全国家实验室数据驱动健康部，卫生服务管理培训中心，塞梅尔维斯大学）

Robert Lakatos, Peter Pollner, Andras Hajdu, Tamas Joo

发表信息：: Mach. Learn. Knowl. Extr. 2025, 7, 15
DOI：: 10.3390/make7010015

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文在多个开源大语言模型和数据集上比较RAG与微调，发现简单RAG架构在ROUGE、BLEU和余弦相似度上显著优于微调模型，并揭示二者结合可能损害性能。

摘要：

生成式大语言模型（G-LLM）的发展为开发类似ChatGPT、Bing或Gemini的新型知识型系统开辟了新机遇。微调（FN）和检索增强生成（RAG）是可用于实现领域适配、以开发基于G-LLM的知识系统的技术。在本研究中，我们使用ROUGE、BLEU、METEOR分数和余弦相似度，比较并考察了RAG和FN在GPT-J-6B、OPT-6.7B、LlaMA、LlaMA-2语言模型上的性能。基于在不同数据集上展示的测量结果，我们证明基于RAG的构建比使用FN生成的模型更高效。我们指出，将RAG与FN结合并非易事，因为将FN模型与RAG连接可能导致性能下降。此外，我们概述了一种简单的基于RAG的架构，平均而言，其在ROGUE分数上比FN模型高出16%，在BLEU分数上高出15%，在余弦相似度上高出53%。这表明RAG在幻觉方面相较FN具有显著优势，而FN模型平均高出8%的METEOR分数所表明的相较RAG更强的创造力，并不能抵消这一优势。

英文摘要：

The development of generative large language models (G-LLM) opened up new opportunities for the development of new types of knowledge-based systems similar to ChatGPT, Bing, or Gemini. Fine-tuning (FN) and Retrieval-Augmented Generation (RAG) are the techniques that can be used to implement domain adaptation for the development of G-LLM-based knowledge systems. In our study, using ROUGE, BLEU, METEOR scores, and cosine similarity, we compare and examine the performance of RAG and FN for the GPT-J-6B, OPT-6.7B, LlaMA, LlaMA-2 language models. Based on measurements shown on different datasets, we demonstrate that RAG-based constructions are more efficient than models produced with FN. We point out that connecting RAG and FN is not trivial, because connecting FN models with RAG can cause a decrease in performance. Furthermore, we outline a simple RAG-based architecture which, on average, outperforms the FN models by 16% in terms of the ROGUE score, 15% in the case of the BLEU score, and 53% based on the cosine similarity. This shows the significant advantage of RAG over FN in terms of hallucination, which is not offset by the fact that the average 8% better METEOR score of FN models indicates greater creativity compared to RAG.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。