通过迭代追问改进医学中的检索增强生成
Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions
AI总结：
提出医学迭代检索增强生成框架i-MedRAG，通过LLMs迭代提出追问并指导后续查询，在USMLE和MMLU等医学问答任务上超越传统RAG，零样本下于MedQA达69.68%准确率。
摘要：
大型语言模型（LLMs）的涌现能力在解决医学问题方面展现了巨大潜力。它们可以掌握大量医学知识，但仍然可能产生幻觉，并且在知识更新上缺乏灵活性。虽然检索增强生成（RAG）被提出用于利用外部知识库增强LLMs的医学问答能力，但在需要多轮信息检索的复杂情况下仍可能失败。为了解决这一问题，我们提出了医学迭代RAG（i-MedRAG），LLMs可以基于先前的信息检索尝试迭代地提出追问。在i-MedRAG的每次迭代中，追问将由传统RAG系统回答，并进一步用于指导下一次迭代的查询生成。我们的实验表明，与美国医师执照考试（USMLE）临床情境中的复杂问题以及大规模多任务语言理解（MMLU）数据集中的各类知识测试相比，i-MedRAG为多种LLMs带来了优于传统RAG的性能提升。值得注意的是，我们的零样本i-MedRAG在GPT-3.5上超越了所有现有的提示工程和微调方法，在MedQA数据集上达到69.68%的准确率。此外，我们还刻画了i-MedRAG在不同追问迭代次数和每次迭代查询数量下的扩展特性。我们的案例研究表明，i-MedRAG能够灵活提出追问以形成推理链，为医学问题提供深入分析。据我们所知，这是首个将追问纳入医学RAG的研究。i-MedRAG的实现可在 https://github.com/Teddy-XiongGZ/MedRAG 获取。
英文摘要：
The emergent abilities of large language models (LLMs) have demonstrated great potential in solving medical questions. They can possess considerable medical knowledge, but may still hallucinate and are inflexible in the knowledge updates. While Retrieval-Augmented Generation (RAG) has been proposed to enhance the medical question-answering capabilities of LLMs with external knowledge bases, it may still fail in complex cases where multiple rounds of information-seeking are required. To address such an issue, we propose iterative RAG for medicine (i-MedRAG), where LLMs can iteratively ask follow-up queries based on previous information-seeking attempts. In each iteration of i-MedRAG, the follow-up queries will be answered by a conventional RAG system and they will be further used to guide the query generation in the next iteration. Our experiments show the improved performance of various LLMs brought by i-MedRAG compared with conventional RAG on complex questions from clinical vignettes in the United States Medical Licensing Examination (USMLE), as well as various knowledge tests in the Massive Multitask Language Understanding (MMLU) dataset. Notably, our zero-shot i-MedRAG outperforms all existing prompt engineering and fine-tuning methods on GPT-3.5, achieving an accuracy of 69.68% on the MedQA dataset. In addition, we characterize the scaling properties of i-MedRAG with different iterations of follow-up queries and different numbers of queries per iteration. Our case studies show that i-MedRAG can flexibly ask follow-up queries to form reasoning chains, providing an in-depth analysis of medical questions. To the best of our knowledge, this is the first-of-its-kind study on incorporating follow-up queries into medical RAG. The implementation of i-MedRAG is available at https://github.com/Teddy-XiongGZ/MedRAG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
