AutoRAG:用于优化检索增强生成流水线的自动化框架
AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline
AI总结:
针对 RAG 模块性能随数据集变化、难以选择的问题,本文提出 AutoRAG 框架,自动为给定数据集探索并逼近最优 RAG 模块组合,并公开实验结果与数据。
摘要:
将 LLM(大语言模型)与外部文档结合使用,已使 RAG(检索增强生成)成为一项关键技术。目前已有大量 RAG 技术和模块正在被研究,但它们在不同数据集上的性能可能存在差异。为特定数据集找到表现良好的 RAG 模块颇具挑战。本文提出 AutoRAG 框架,它能针对给定数据集自动识别合适的 RAG 模块。AutoRAG 会为该数据集探索并逼近 RAG 模块的最优组合。此外,我们还分享了使用 AutoRAG 优化一个数据集的结果。所有实验结果和数据均公开发布,可通过 GitHub 仓库 https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper 获取。
英文摘要:
Using LLMs (Large Language Models) in conjunction with external documents has made RAG (Retrieval-Augmented Generation) an essential technology. Numerous techniques and modules for RAG are being researched, but their performance can vary across different datasets. Finding RAG modules that perform well on specific datasets is challenging. In this paper, we propose the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. Additionally, we share the results of optimizing a dataset using AutoRAG. All experimental results and data are publicly available and can be accessed through our GitHub repository https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。