RiskAgent: 语言模型与验证工具协同以基于证据的风险预测
RiskAgent: Synergizing Language Models with Validated Tools for Evidence-Based Risk Prediction
AI总结:
RiskAgent通过整合语言模型与验证的临床决策工具,实现基于证据的风险预测,展现优越的性能和泛化能力。
摘要:
大型语言模型(LLMs)在医学考试中与人类专家竞争取得竞争性结果。然而,将其应用于复杂的临床决策仍是一个挑战,这需要深入理解医学知识,不同于当前常用的标准化考试式场景。一种常见方法是针对目标任务微调LLMs,但这种方法不仅需要大量数据和计算资源,而且仍然容易生成“幻觉”。在本工作中,我们提出了RiskAgent,它将语言模型与数百种由循证医学支持的临床决策工具协同起来,以提供可推广且忠实的推荐。我们的实验表明,RiskAgent不仅在广泛的临床风险预测上实现了优越的性能,涵盖了多样化的场景和疾病,而且在外部MedCalc-Bench数据集上的工具学习中也表现出鲁棒的泛化能力,并在三个代表性基准上展示了医学推理和问答的能力,即MedQA、MedMCQA和MMLU。
英文摘要:
Large Language Models (LLMs) achieve competitive results compared to human experts in medical examinations. However, it remains a challenge to apply LLMs to complex clinical decision-making, which requires a deep understanding of medical knowledge and differs from the standardized, exam-style scenarios commonly used in current efforts. A common approach is to fine-tune LLMs for target tasks, which, however, not only requires substantial data and computational resources but also remains prone to generating `hallucinations'. In this work, we present RiskAgent, which synergizes language models with hundreds of validated clinical decision tools supported by evidence-based medicine, to provide generalizable and faithful recommendations. Our experiments show that RiskAgent not only achieves superior performance on a broad range of clinical risk predictions across diverse scenarios and diseases, but also demonstrates robust generalization in tool learning on the external MedCalc-Bench dataset, as well as in medical reasoning and question answering on three representative benchmarks, MedQA, MedMCQA, and MMLU.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。