MRAG：生物医学领域检索增强生成的基准测试
MRAG: Benchmarking Retrieval-Augmented Generation for Bio-medicine
AI总结：
MRAG基准测试通过构建医学领域语料库和工具包，评估RAG在生物医学任务中的有效性，揭示RAG在提升LLM可靠性及性能影响因素方面的贡献。
摘要：
尽管检索增强生成（RAG）已被迅速应用于科学和临床问答系统中，但在医疗领域缺乏全面的评估基准。为解决这一缺口，我们引入了医学检索增强生成（MRAG）基准，涵盖英语和中文的各种任务，并构建了包含维基百科和PubMed的语料库。此外，我们开发了MRAG-Toolkit，以系统地探索不同的RAG组件。我们的实验揭示了以下发现：（a）RAG在MRAG任务中增强了LLM的可靠性。（b）RAG系统的性能受检索方法、模型大小和提示策略的影响。（c）尽管RAG提高了有用性和推理质量，但LLM的响应可能在长形式问题中变得略微不那么易读。我们将在接受后以CCBY-4.0许可证发布MRAG-Bench的数据集和工具包，以促进学术界和工业界的应用。
英文摘要：
While Retrieval-Augmented Generation (RAG) has been swiftly adopted in scientific and clinical QA systems, a comprehensive evaluation benchmark in the medical domain is lacking. To address this gap, we introduce the Medical Retrieval-Augmented Generation (MRAG) benchmark, covering various tasks in English and Chinese languages, and building a corpus with Wikipedia and Pubmed. Additionally, we develop the MRAG-Toolkit, facilitating systematic exploration of different RAG components. Our experiments reveal that: (a) RAG enhances LLM reliability across MRAG tasks. (b) the performance of RAG systems is influenced by retrieval approaches, model sizes, and prompting strategies. (c) While RAG improves usefulness and reasoning quality, LLM responses may become slightly less readable for long-form questions. We will release the MRAG-Bench's dataset and toolkit with CCBY-4.0 license upon acceptance, to facilitate applications from both academia and industry.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
