Case-based reasoning (CBR) is an experience-based approach to problem solving, where a repository of solved cases is adapted to solve new cases. Recent research shows that Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) can support the Retrieve and Reuse stages of the CBR pipeline by retrieving similar cases and using them as additional context to an LLM query. Most studies have focused on text-only applications, however, in many real-world problems the components of a case are multimodal. In this paper we present MCBR-RAG, a general RAG framework for multimodal CBR applications. The MCBR-RAG framework converts non-text case components into text-based representations, allowing it to: 1) learn application-specific latent representations that can be indexed for retrieval, and 2) enrich the query provided to the LLM by incorporating all case components for better context. We demonstrate MCBR-RAG's effectiveness through experiments conducted on a simplified Math-24 application and a more complex Backgammon application. Our empirical results show that MCBR-RAG improves generation quality compared to a baseline LLM with no contextual information provided.
Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation
A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
Department of Mechatronics & Industrial Engineering, Chittagong University of Engineering & Technology(吉大港工程与技术大学机电与工业工程系);Department of Mechanical Engineering, Chittagong University of Engineering & Technology(吉大港工程与技术大学机械工程系)
MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering
Key Laboratory of Target Cognition and Application Technology (TCAT), AIRI, CAS(目标认知与应用技术重点实验室(TCAT),空气动力研究所,中国科学院);School of Electronic, Electrical and Communication Engineering, UCAS(电子、电气与通信工程学院,中国科学院大学);Aerospace Information Research Institute, Chinese Academy of Sciences(航天信息研究所,中国科学院);Baidu Inc.(百度公司)
R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generation
School of Computer Science, South China Normal University(华南师范大学计算机学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学)
Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Artificial Intelligence Laboratory, China Mobile Communication Group Tianjin Co., Ltd.(中国移动通信集团天津有限公司人工智能实验室);University College London(伦敦大学学院)
DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation
Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University(浙江智能教育技术与应用重点实验室,浙江师范大学);School of Computer Science and Technology, Zhejiang Normal University(浙江师范大学计算机科学与技术学院);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Department of Computer Science and Engineering, Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院)
RAR$^2$: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval
Department of Computing, The Hong Kong Polytechnic University(计算机系,香港理工大学);Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology(可信自主系统研究院和计算机科学与工程系,南方科技大学)
BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generation
University of Science and Technology of China(中国科学技术大学);Unisound AI Technology Co.Ltd(Unisound AI技术有限公司);MAIS, Institute of Automation, Chinese Academy of Sciences(自动化研究所,中国科学院)
ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cybersecurity, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Alibaba Group(阿里巴巴集团)
RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China(语音与语言信息处理国家工程研究中心,中国科学技术大学);State Key Laboratory of Cognitive Intelligence, iFLYTEK Research(认知智能国家重点实验室,iFLYTEK研究院)