Retrieval Augmented Generation (RAG) systems have seen huge popularity in augmenting Large-Language Model (LLM) outputs with domain specific and time sensitive data. Very recently a shift is happening from simple RAG setups that query a vector database for additional information with every user input to more sophisticated forms of RAG. However, different concrete approaches compete on mostly anecdotal evidence at the moment. In this paper we present a rigorous dataset creation and evaluation workflow to quantitatively compare different RAG strategies. We use a dataset created this way for the development and evaluation of a boolean agent RAG setup: A system in which a LLM can decide whether to query a vector database or not, thus saving tokens on questions that can be answered with internal knowledge. We publish our code and generated dataset online.
Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries
Yale University(耶鲁大学);University of Pennsylvania(宾夕法尼亚大学);Human Language Technology Center of Excellence(语言技术中心卓越中心);Johns Hopkins University(约翰霍普金斯大学)
Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);LG AI Research(LG AI研究院);Cornell University(康奈尔大学);Korea University(高丽大学);University of Illinois Chicago(芝加哥伊利诺伊大学)
Grounding Large Language Models in Clinical Evidence: A Retrieval-Augmented Generation System for Querying UK NICE Clinical Guidelines
Institute of Health Informatics, University College London(健康信息学研究所,伦敦大学学院);Department of Biostatistics and Health Informatics, King’s College London(生物统计学与健康信息学系,伦敦国王学院);EPSRC DRIVE-Health CDT, London, U.K.(EPSRC DRIVE-Health CDT,伦敦,英国);Imperial College Healthcare NHS Trust(帝国学院医疗 NHS 信托);Black Country Healthcare NHS Foundation Trust(黑斯廷斯地区医疗 NHS 基础信托);Feldon Lane Surgery, The Dudley Group NHS Foundation Trust(费尔顿车道诊所,德比集团 NHS 基础信托);The Cleveland Clinic, London, U.K.(克利夫兰诊所,伦敦,英国);CogStack Limited(CogStack 有限公司)