arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.11929cs.LG
收藏

ALoFTRAG:检索增强生成的自动局部微调

ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation

Lightblue KK

Peter Devine

AI总结:

ALoFTRAG通过生成合成数据并利用LoRA微调,无需人工标注或教师模型,在26种语言的20个数据集上平均提升引用准确率8.3%、回答准确率3.0%,为医疗和金融等敏感领域提供经济且数据安全的RAG精度改进方案。

摘要:

检索增强生成(RAG)系统已被证明能够提高大语言模型(LLM)输出的准确性。然而,当这些模型应用于新的数据领域时,其准确性往往较低。我们引入了检索增强生成模型的自动局部微调(ALoFTRAG)框架,旨在通过训练LLM来提高RAG系统在特定领域的准确性,而无需人工标注数据或使用更大的教师模型。通过生成和过滤合成训练数据并执行LoRA微调,ALoFTRAG在26种语言的20个数据集上平均将引用准确性和回答准确性分别提高了8.3%和3.0%。我们的结果表明,ALoFTRAG为提升RAG准确性提供了一种实用、经济高效且数据安全的解决方案,使其特别适用于医疗和金融等敏感领域。

英文摘要:

Retrieval Augmented Generation (RAG) systems have been shown to improve the accuracy of Large Language Model (LLM) outputs. However, these models can often achieve low accuracy when applied to new data domains. We introduce the Automatic Local Fine Tuning of Retrieval Augmented Generation models (ALoFTRAG) framework, designed to improve the accuracy of RAG systems on a given domain by training LLMs without manually labeled data or using larger teacher models. By generating and filtering synthetic training data and performing LoRA fine-tuning, ALoFTRAG improves citation and answer accuracy across 20 datasets in 26 languages by, on average, 8.3% and 3.0% respectively. Our results demonstrate that ALoFTRAG offers a practical, cost-effective, and data-secure solution for improving RAG accuracy, making it particularly applicable to sensitive domains such as healthcare and finance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑