arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14158cs.CLcs.AIcs.MA
收藏

面向医疗AI的多智能体框架:利用微调的GPT、LLaMA和DeepSeek R1进行基于证据和偏见意识的临床查询处理

A Multi-Agent Framework for Medical AI: Leveraging Fine-Tuned GPT, LLaMA, and DeepSeek R1 for Evidence-Based and Bias-Aware Clinical Query Processing

Department of Computing;Games, Teesside University, Middlesbrough, United Kingdom Centre for Digital Innovation, Teesside University, Middlesbrough, United Kingdom Faculty of Business \& Technology, University of Sunderland, Sunderland, United Kingdom

Naeimeh Nourmohammadi, Md Meem Hossain, The Anh Han, Safina Showkat Ara, Zia Ush Shamszaman

AI总结:

本文提出一个多智能体框架,利用微调的GPT、LLaMA和DeepSeek R1,结合证据检索和偏见检查,提升医疗问答的可靠性与准确性。

摘要:

大型语言模型(LLMs)在医疗问答中显示出潜力,但临床应用受限于验证能力弱、证据支撑不足和信心信号不可靠。我们提出一个多智能体医疗问答框架,结合互补的LLMs与证据检索、不确定性估计和偏见检查,以提高回答可靠性。我们的方法分为两个阶段。首先,我们对三种代表性的LLM家族(GPT、LLaMA和DeepSeek R1)在MedQuAD衍生的医疗问答数据(超过2万对问题-答案对,涵盖多个NIH领域)上进行微调,并进行生成质量的基准测试。DeepSeek R1在ROUGE-1(0.536±0.04)、ROUGE-2(0.226±0.03)和BLEU(0.098±0.018)方面得分最高,并在零样本评估中大幅优于专门的生物医学基线模型BioGPT。其次,我们实现了一个模块化的多智能体流水线,其中临床推理代理(微调的LLaMA)生成结构化解释,证据检索代理查询PubMed以将响应根植于近期文献,而精修代理(DeepSeek R1)提高清晰度和事实一致性;对于高风险或高不确定性情况,可触发可选的人类验证路径。安全机制包括蒙特卡洛dropout和基于困惑度的不确定性评分,以及通过LIME/SHAP分析支持的词汇和情感偏见检测。在评估中,完整系统实现了87%的准确率,相关性约为0.80,证据增强将不确定性(困惑度4.13)降低到基线响应水平,平均端到端延迟为36.5秒(在报告的配置下)。总体而言,结果表明,智能体专业化和验证层可以缓解关键单模型限制,并提供一个实用且可扩展的设计方案,用于基于证据和偏见意识的医疗AI。

英文摘要:

Large language models (LLMs) show promise for healthcare question answering, but clinical use is limited by weak verification, insufficient evidence grounding, and unreliable confidence signalling. We propose a multi-agent medical QA framework that combines complementary LLMs with evidence retrieval, uncertainty estimation, and bias checks to improve answer reliability. Our approach has two phases. First, we fine-tune three representative LLM families (GPT, LLaMA, and DeepSeek R1) on MedQuAD-derived medical QA data (20k+ question-answer pairs across multiple NIH domains) and benchmark generation quality. DeepSeek R1 achieves the strongest scores (ROUGE-1 0.536 +- 0.04; ROUGE-2 0.226 +-0.03; BLEU 0.098 -+ 0.018) and substantially outperforms the specialised biomedical baseline BioGPT in zero-shot evaluation. Second, we implement a modular multi-agent pipeline in which a Clinical Reasoning agent (fine-tuned LLaMA) produces structured explanations, an Evidence Retrieval agent queries PubMed to ground responses in recent literature, and a Refinement agent (DeepSeek R1) improves clarity and factual consistency; an optional human validation path is triggered for high-risk or high-uncertainty cases. Safety mechanisms include Monte Carlo dropout and perplexity-based uncertainty scoring, plus lexical and sentiment-based bias detection supported by LIME/SHAP-based analyses. In evaluation, the full system achieves 87% accuracy with relevance around 0.80, and evidence augmentation reduces uncertainty (perplexity 4.13) compared to base responses, with mean end-to-end latency of 36.5 seconds under the reported configuration. Overall, the results indicate that agent specialisation and verification layers can mitigate key single-model limitations and provide a practical, extensible design for evidence-based and bias-aware medical AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑