arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.00744cs.DBcs.CLcs.IR
收藏

CARROT:一种用于RAG的学得式成本约束检索优化系统

CARROT: A Learned Cost-Constrained Retrieval Optimization System for RAG

Ziting Wang, Haitao Yuan, Wei Dong, Gao Cong, Feifei Li

AI总结:

CARROT通过MCTS和配置代理优化RAG检索,提升检索效率与准确性,实验显示性能提升达30%。

摘要:

大型语言模型(LLMs)在生成和推理任务中表现出色,但在处理最新知识方面存在困难,导致不准确或幻觉。检索增强生成(RAG)通过检索并将其纳入输入提示来缓解这一问题。特别是由于LLMs的上下文窗口限制和长上下文幻觉,只有最相关的“块”被检索。然而,当前的RAG系统面临三个关键挑战:(1)块通常是独立检索的,没有考虑它们之间的关系,如冗余和顺序;(2)块的效用是非单调的,添加更多块可能会降低质量;(3)检索策略无法适应不同查询的独特特性。为克服这些挑战,我们设计了一种用于RAG的成本约束检索优化框架。我们采用基于蒙特卡洛树搜索(MCTS)的策略来寻找最佳块组合顺序,该策略考虑块之间的相关性。此外,为了应对块效用的非单调性,我们设计了一种效用计算策略,以识别最佳块组合,而无需 necessarily耗尽预算。此外,我们提出了一种配置代理,预测每个查询领域的最佳配置,提高我们框架的适应性和效率。实验结果表明,与基线模型相比,性能提高了高达30%,突显了该框架的有效性、可扩展性和适用性。我们的源代码已发布在https://github.com/wang0702/CARROT。

英文摘要:

Large Language Models (LLMs) have demonstrated impressive ability in generation and reasoning tasks but struggle with handling up-to-date knowledge, leading to inaccuracies or hallucinations. Retrieval-Augmented Generation (RAG) mitigates this by retrieving and incorporating external knowledge into input prompts. In particular, due to LLMs' context window limitations and long-context hallucinations, only the most relevant "chunks" are retrieved. However, current RAG systems face three key challenges: (1) chunks are often retrieved independently without considering their relationships, such as redundancy and ordering; (2) the utility of chunks is non-monotonic, as adding more chunks can degrade quality; and (3) retrieval strategies fail to adapt to the unique characteristics of different queries. To overcome these challenges, we design a cost-constrained retrieval optimization framework for RAG. We adopt a Monte Carlo Tree Search (MCTS) based strategy to find the optimal chunk combination order, which considers the chunks' correlations. In addition, to address the non-monotonicity of chunk utility, instead of treating budget exhaustion as the termination condition, we design a utility computation strategy to identify the optimal chunk combination without necessarily exhausting the budget. Furthermore, we propose a configuration agent that predicts optimal configurations for each query domain, improving our framework's adaptability and efficiency. Experimental results demonstrate up to a 30% improvement over baseline models, highlighting the framework's effectiveness, scalability, and suitability. Our source code has been released at https://github.com/wang0702/CARROT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑