arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.05030cs.AIcs.CL
收藏

面向多模态基于案例推理应用的通用检索增强生成框架

A General Retrieval-Augmented Generation Framework for Multimodal Case-Based Reasoning Applications

Ofir Marom

AI总结:

本文提出MCBR-RAG通用框架,将多模态案例组件转为文本表示以支持检索和LLM上下文增强,并在Math-24与Backgammon应用中验证其可提升生成质量。

摘要:

基于案例推理(CBR)是一种基于经验的问题解决方法,其中已解决案例的知识库会被调整以解决新案例。近期研究表明,结合检索增强生成(RAG)的大型语言模型(LLM)能够通过检索相似案例并将其作为LLM查询的附加上下文,支持CBR流程的检索和复用阶段。然而,大多数研究集中于纯文本应用,而在许多现实问题中,案例的组成部分是多模态的。本文提出MCBR-RAG,这是一个面向多模态CBR应用的通用RAG框架。MCBR-RAG框架将非文本案例组成部分转换为基于文本的表示,使其能够:1)学习可被索引用于检索的、特定于应用的潜在表示;2)通过纳入所有案例组成部分来丰富提供给LLM的查询,以获得更好的上下文。我们通过在简化的Math-24应用和更复杂的双陆棋(Backgammon)应用上进行的实验,证明了MCBR-RAG的有效性。实验结果表明,与未提供上下文信息的基线LLM相比,MCBR-RAG提升了生成质量。

英文摘要:

Case-based reasoning (CBR) is an experience-based approach to problem solving, where a repository of solved cases is adapted to solve new cases. Recent research shows that Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) can support the Retrieve and Reuse stages of the CBR pipeline by retrieving similar cases and using them as additional context to an LLM query. Most studies have focused on text-only applications, however, in many real-world problems the components of a case are multimodal. In this paper we present MCBR-RAG, a general RAG framework for multimodal CBR applications. The MCBR-RAG framework converts non-text case components into text-based representations, allowing it to: 1) learn application-specific latent representations that can be indexed for retrieval, and 2) enrich the query provided to the LLM by incorporating all case components for better context. We demonstrate MCBR-RAG's effectiveness through experiments conducted on a simplified Math-24 application and a more complex Backgammon application. Our empirical results show that MCBR-RAG improves generation quality compared to a baseline LLM with no contextual information provided.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑