arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.00029cs.CLcs.AI
收藏

聚类检索增强生成 (CRAG)

Clustered Retrieved Augmented Generation (CRAG)

AI Center of Excellence(卓越人工智能中心);Sinch AB(辛奇公司)

Simon Akesson, Frances A. Santos

AI总结:

针对RAG因检索上下文过长导致成本高和延迟大的问题,提出CRAG方法,通过聚类有效减少提示token数量(最高超90%)且不降低响应质量。

摘要:

向大型语言模型(LLMs)提供外部知识是在实际应用中使用这些模型的关键,原因包括:以实时方式整合最新内容、提供对特定领域知识的访问,以及有助于防止幻觉。为此,基于向量数据库的Retrieval Augmented Generation (RAG)方法已被广泛采用。因此,外部知识的任何部分都可以被检索并作为输入上下文提供给某些LLM。尽管RAG方法取得了成功,但它对某些应用仍可能不可行,因为检索到的上下文可能需要比LLM支持的长度更长的上下文窗口。即使检索到的上下文适合上下文窗口大小,token数量也可能很庞大,从而影响成本和处理时间,对大多数应用变得不切实际。为了解决这些问题,我们提出了CRAG,这是一种新颖的方法,与使用RAG的解决方案相比,能够有效减少提示token数量而不降低生成响应的质量。通过我们的实验,我们表明与RAG相比,CRAG可将token数量减少至少46%,在某些情况下超过90%。此外,当分析的评论数量增加时,CRAG的token数量不会显著增加,而RAG则不然,当有75条评论时,其token数量几乎是4条评论时的9倍。

英文摘要:

Providing external knowledge to Large Language Models (LLMs) is a key point for using these models in real-world applications for several reasons, such as incorporating up-to-date content in a real-time manner, providing access to domain-specific knowledge, and contributing to hallucination prevention. The vector database-based Retrieval Augmented Generation (RAG) approach has been widely adopted to this end. Thus, any part of external knowledge can be retrieved and provided to some LLM as the input context. Despite RAG approach's success, it still might be unfeasible for some applications, because the context retrieved can demand a longer context window than the size supported by LLM. Even when the context retrieved fits into the context window size, the number of tokens might be expressive and, consequently, impact costs and processing time, becoming impractical for most applications. To address these, we propose CRAG, a novel approach able to effectively reduce the number of prompting tokens without degrading the quality of the response generated compared to a solution using RAG. Through our experiments, we show that CRAG can reduce the number of tokens by at least 46\%, achieving more than 90\% in some cases, compared to RAG. Moreover, the number of tokens with CRAG does not increase considerably when the number of reviews analyzed is higher, unlike RAG, where the number of tokens is almost 9x higher when there are 75 reviews compared to 4 reviews.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑