人机协同推理用于临床诊断:证据整合语言代理
Human-AI Co-reasoning for Clinical Diagnosis with Evidence-Integrated Language Agent
AI总结:
本文提出PULSE,结合领域调优的大语言模型与文献检索,支持复杂病例的诊断决策。实验显示PULSE在专家级准确度上表现优异,且在罕见病病例中保持稳定性能,揭示了语言模型在临床诊断中的潜力与局限。
摘要:
我们提出了PULSE,一种结合领域调优的大语言模型与科学文献检索的医学推理代理,以支持复杂现实病例中的诊断决策制定。为了评估其能力,我们编制了一个包含82个真实内分泌学病例报告的基准测试集,涵盖了广泛疾病类型和发病率级别。在受控实验中,我们比较了PULSE与不同专业水平的医生(从住院医师到高级专家)的性能,并考察了AI辅助如何影响人类诊断推理。PULSE在专家级准确度上表现优异,超越住院医师和初级专家,同时在Top@1和Top@4阈值上与高级专家表现相当。与医生不同,医生的准确性随疾病罕见性下降,而PULSE在不同发病率层级中保持稳定性能。该代理还表现出适应性推理,随着病例难度增加,输出长度相应增加,类似于专家临床医生的更长时间 deliberation。当协作使用时,PULSE使医生能够纠正初始错误并扩展诊断假设,但也引入了自动化偏差的风险。本研究探讨了串行和并行协作工作流,揭示了PULSE在常见和罕见病例中的稳健支持。这些发现突显了基于语言模型的代理在临床诊断中的潜力与局限,并提供了一个评估其在现实决策中的作用的框架。
英文摘要:
We present PULSE, a medical reasoning agent that combines a domain-tuned large language model with scientific literature retrieval to support diagnostic decision-making in complex real-world cases. To evaluate its capabilities, we curated a benchmark of 82 authentic endocrinology case reports encompassing a broad spectrum of disease types and incidence levels. In controlled experiments, we compared PULSE's performance against physicians with varying levels of expertise-from residents to senior specialists-and examined how AI assistance influenced human diagnostic reasoning. PULSE attained expert-competitive accuracy, outperforming residents and junior specialists while matching senior specialist performance at both Top@1 and Top@4 thresholds. Unlike physicians, whose accuracy declined with disease rarity, PULSE maintained stable performance across incidence tiers. The agent also exhibited adaptive reasoning, increasing output length with case difficulty in a manner analogous to the longer deliberation observed among expert clinicians. When used collaboratively, PULSE enabled physicians to correct initial errors and broaden diagnostic hypotheses, but also introduced risks of automation bias. The study explores both serial and concurrent collaboration workflows, revealing that PULSE offers robust support across common and rare presentations. These findings underscore both the promise and the limitations of language model-based agents in clinical diagnosis, and offer a framework for evaluating their role in real-world decision-making.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。