MedRAGChecker: 用于生物医学检索增强生成的声明级验证
MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation
AI总结：
MedRAGChecker通过声明级验证和诊断框架，提高生物医学RAG生成答案的可靠性与安全性。
摘要：
生物医学检索增强生成（RAG）可以将大语言模型的答案 grounded 在医学文献中，但长格式输出往往包含孤立的未支持或矛盾的声明，这具有安全影响。我们介绍了MedRAGChecker，一种用于生物医学RAG的声明级验证和诊断框架。给定一个问题、检索到的证据和生成的答案，MedRAGChecker将答案分解为原子声明，并通过结合证据基础的自然语言推理（NLI）与生物医学知识图谱（KG）一致性信号来估计声明的支持情况。汇总声明决策可产生答案级诊断，帮助区分检索和生成失败，包括忠实性、证据不足、矛盾和安全关键错误率。为了实现可扩展评估，我们将流程精简为紧凑的生物医学模型，并使用具有类特定可靠性加权的集成验证器。在四个生物医学问答基准上的实验表明，MedRAGChecker能够可靠地标记未支持和矛盾的声明，并揭示不同生成器在安全关键生物医学关系上的不同风险特征。
英文摘要：
Biomedical retrieval-augmented generation (RAG) can ground LLM answers in medical literature, yet long-form outputs often contain isolated unsupported or contradictory claims with safety implications. We introduce MedRAGChecker, a claim-level verification and diagnostic framework for biomedical RAG. Given a question, retrieved evidence, and a generated answer, MedRAGChecker decomposes the answer into atomic claims and estimates claim support by combining evidence-grounded natural language inference (NLI) with biomedical knowledge-graph (KG) consistency signals. Aggregating claim decisions yields answer-level diagnostics that help disentangle retrieval and generation failures, including faithfulness, under-evidence, contradiction, and safety-critical error rates. To enable scalable evaluation, we distill the pipeline into compact biomedical models and use an ensemble verifier with class-specific reliability weighting. Experiments on four biomedical QA benchmarks show that MedRAGChecker reliably flags unsupported and contradicted claims and reveals distinct risk profiles across generators, particularly on safety-critical biomedical relations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
