arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.08438cs.AI
收藏

面向学术数据的大语言模型检索增强生成优化研究

Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data

Technical University of Munich(慕尼黑工业大学)

Anum Afzal, Juraj Vladika, Gentrit Fazlija, Andrei Staradubets, Florian Matthes

AI总结:

本文针对学术领域数据,测试并整合了Multi-Query等四种优化技术于RAG框架,引入RAG混淆矩阵评估配置,并比较了多种开源与闭源大语言模型,发现检索阶段引入多查询显著提升了性能。

摘要:

鉴于越来越多组织将检索增强生成(RAG)整合到其运营中的趋势,我们在特定领域数据上评估了RAG,并测试了最先进模型在各种优化技术下的表现。我们引入了四种优化方法:Multi-Query、Child-Parent-Retriever、Ensemble Retriever和In-Context-Learning,以增强其在学术领域的功能和性能。我们专注于数据检索,特别是针对一所大型技术大学的各个学习项目。此外,我们还引入了一种新颖的评估方法——RAG混淆矩阵,旨在评估RAG框架内各种配置的有效性。通过探索开源(如Llama2、Mistral)和闭源(GPT-3.5和GPT-4)大语言模型的整合,我们为RAG框架在特定领域背景下的应用和优化提供了有价值的见解。实验表明,在检索阶段引入多查询时,性能有显著提升。

英文摘要:

Given the growing trend of many organizations integrating Retrieval Augmented Generation (RAG) into their operations, we assess RAG on domain-specific data and test state-of-the-art models across various optimization techniques. We incorporate four optimizations; Multi-Query, Child-Parent-Retriever, Ensemble Retriever, and In-Context-Learning, to enhance the functionality and performance in the academic domain. We focus on data retrieval, specifically targeting various study programs at a large technical university. We additionally introduce a novel evaluation approach, the RAG Confusion Matrix designed to assess the effectiveness of various configurations within the RAG framework. By exploring the integration of both open-source (e.g., Llama2, Mistral) and closed-source (GPT-3.5 and GPT-4) Large Language Models, we offer valuable insights into the application and optimization of RAG frameworks in domain-specific contexts. Our experiments show a significant performance increase when including multi-query in the retrieval phase.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑