arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.15364cs.CLcs.AIcs.IR
收藏

VERA:检索增强系统的验证与增强

VERA: Validation and Enhancement for Retrieval Augmented systems

Enkrypt AI(Enkrypt人工智能公司)

Nitin Aravind Birur, Tanay Baswa, Divyanshu Kumar, Jatan Loya, Sahil Agarwal, Prashanth Harshangi

AI总结:

针对检索增强生成(RAG)中因依赖无关文档或错误推断导致的准确性问题,提出VERA系统,通过在响应前后进行上下文评估优化与原子陈述验证,有效减少幻觉并提升各类大小模型的响应准确性与可靠性。

摘要:

大型语言模型(LLMs)展现出卓越的能力,但往往由于仅依赖其内部嵌入的知识而产生不准确的响应。检索增强生成(RAG)通过引入外部信息检索系统来增强LLMs,在查询之外提供额外的上下文,以减轻特定上下文下的不准确问题。然而,准确性问题依然存在,因为模型可能依赖不相关的文档,或从其训练知识中进行错误的推断。为了评估和改进RAG框架中检索系统和LLM的性能,我们提出了VERA(Validation and Enhancement for Retrieval Augmented systems,检索增强系统的验证与增强),该系统旨在:1)在生成响应之前评估并增强检索到的上下文;2)评估并优化LLM生成的响应,以确保精确性并最小化错误。VERA采用了一个兼具评估与增强功能的LLM,它首先检查是否需要外部检索,评估检索上下文的相关性和冗余度,并对其进行优化以消除非必要信息。在响应生成之后,VERA将响应拆分为原子陈述,评估它们与查询的相关性,并确保其遵循上下文。我们的实验表明,VERA不仅在提升较小的开源模型性能方面表现出显著效果,对大型最先进模型同样如此。这些增强突显了VERA生成准确且相关响应的潜力,推动了检索增强语言建模领域的最新进展。VERA结合了多个评估和优化步骤的稳健方法,有效缓解了幻觉现象并改进了检索和响应过程,使其成为在信息生成中要求高准确性和可靠性的应用的宝贵工具。

英文摘要:

Large language models (LLMs) exhibit remarkable capabilities but often produce inaccurate responses, as they rely solely on their embedded knowledge. Retrieval-Augmented Generation (RAG) enhances LLMs by incorporating an external information retrieval system, supplying additional context along with the query to mitigate inaccuracies for a particular context. However, accuracy issues still remain, as the model may rely on irrelevant documents or extrapolate incorrectly from its training knowledge. To assess and improve the performance of both the retrieval system and the LLM in a RAG framework, we propose \textbf{VERA} (\textbf{V}alidation and \textbf{E}nhancement for \textbf{R}etrieval \textbf{A}ugmented systems), a system designed to: 1) Evaluate and enhance the retrieved context before response generation, and 2) Evaluate and refine the LLM-generated response to ensure precision and minimize errors. VERA employs an evaluator-cum-enhancer LLM that first checks if external retrieval is necessary, evaluates the relevance and redundancy of the retrieved context, and refines it to eliminate non-essential information. Post-response generation, VERA splits the response into atomic statements, assesses their relevance to the query, and ensures adherence to the context. Our experiments demonstrate VERA's remarkable efficacy not only in improving the performance of smaller open-source models, but also larger state-of-the art models. These enhancements underscore VERA's potential to produce accurate and relevant responses, advancing the state-of-the-art in retrieval-augmented language modeling. VERA's robust methodology, combining multiple evaluation and refinement steps, effectively mitigates hallucinations and improves retrieval and response processes, making it a valuable tool for applications demanding high accuracy and reliability in information generation. .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑