arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.09252cs.CLcs.IR
收藏

用于RAG中高效答案生成的上下文嵌入

Context Embeddings for Efficient Answer Generation in RAG

University of Amsterdam(阿姆斯特丹大学);Naver Labs Europe(Naver欧洲实验室);The University of Queensland(昆士兰大学)

David Rau, Shuai Wang, Hervé Déjean, Stéphane Clinchant

AI总结:

针对RAG中长上下文拖慢LLM解码速度的问题,提出上下文压缩方法COCOM,将长上下文压缩为少量上下文嵌入,可灵活调节压缩率,相比现有方法最高提速5.69倍且性能更优。

摘要:

检索增强生成(Retrieval-Augmented Generation,RAG)通过外部信息扩展输入,得以解决大语言模型(LLMs)知识有限的问题。但这会导致模型的上下文输入变得更长,直接拖慢解码速度,最终转化为用户等待答案的时长。我们针对这一挑战提出了COCOM——一种高效的上下文压缩方法,它将长上下文压缩为仅少量Context Embeddings(上下文嵌入),大幅加快生成速度。该方法支持多种压缩率,可在解码时间与答案质量间做权衡。与现有方法相比,COCOM能更高效地处理多上下文,显著缩短长输入的解码时间。实验显示,相比现有高效上下文压缩方法,COCOM实现了最高5.69倍的速度提升,同时取得了更优的性能。

英文摘要:

Retrieval-Augmented Generation (RAG) allows overcoming the limited knowledge of LLMs by extending the input with external information. As a consequence, the contextual inputs to the model become much longer which slows down decoding time directly translating to the time a user has to wait for an answer. We address this challenge by presenting COCOM, an effective context compression method, reducing long contexts to only a handful of Context Embeddings speeding up the generation time by a large margin. Our method allows for different compression rates trading off decoding time for answer quality. Compared to earlier methods, COCOM allows for handling multiple contexts more effectively, significantly reducing decoding time for long inputs. Our method demonstrates a speed-up of up to 5.69 $\times$ while achieving higher performance compared to existing efficient context compression methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑