检索增强生成方法的超参数优化分析
An Analysis of Hyper-Parameter Optimization Methods for Retrieval Augmented Generation
AI总结：
本文分析了RAG超参数优化方法，通过五种算法和五种数据集的实验，发现贪婪或随机搜索能高效提升RAG性能，且优先优化模型选择优于传统流程顺序。
摘要：
对特定任务进行检索增强生成（RAG）配置的优化是一个复杂且资源密集的挑战。受此挑战的启发，最近出现了针对RAG超参数优化（HPO）的框架，但其有效性尚未经过严格基准测试。为填补这一空白，我们进行了涵盖五个HPO算法和五个来自不同领域的数据集的综合研究，包括一个新整理的现实世界产品文档数据集。我们的研究探索了迄今为止最大的RAG HPO搜索空间，其中包括完整的网格搜索评估，并使用三个评估指标作为优化目标。分析结果表明，RAG HPO可以高效地进行，无论是贪婪搜索还是随机搜索，并且对所有数据集都显著提升了RAG性能。对于贪婪HPO方法，我们表明在优化过程中首先优化模型选择优于通常遵循RAG流程顺序的做法。
英文摘要：
Optimizing Retrieval-Augmented Generation (RAG) configurations for specific tasks is a complex and resource-intensive challenge. Motivated by this challenge, frameworks for RAG hyper-parameter optimization (HPO) have recently emerged, yet their effectiveness has not been rigorously benchmarked. To fill this gap, we present a comprehensive study involving five HPO algorithms over five datasets from diverse domains, including a newly curated real-world product documentation dataset. Our study explores the largest RAG HPO search space to date that includes full grid-search evaluations, and uses three evaluation metrics as optimization targets. Analysis of the results shows that RAG HPO can be done efficiently, either greedily or with random search, and that it significantly boosts RAG performance for all datasets. For greedy HPO approaches, we show that optimizing model selection first is preferable to the common practice of following the RAG pipeline order during optimization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
