在虚拟临床环境中进化交互诊断代理
Evolving Interactive Diagnostic Agents in a Virtual Clinical Environment
AI总结：
本研究提出DiagAgent，通过强化学习在虚拟临床环境中训练，实现多轮交互式诊断，显著提升诊断准确性和检查推荐效果。
摘要：
我们提出了一种框架，用于通过强化学习训练大型语言模型（LLMs）作为诊断代理，使其能够管理多轮交互式诊断过程，自适应地选择检查项目并最终做出诊断。与基于静态数据训练的指令调优模型不同，我们的方法通过动态探索和基于结果的反馈获取诊断策略，将不断演变的患者状态映射到下一步最优的检查和后续诊断。我们的贡献包括：（i）DiagGym，一个基于电子健康记录训练的诊断世界模型，作为支持交互诊断闭环硅仿真训练和评估的虚拟临床环境；（ii）DiagAgent，通过端到端多轮强化学习训练，学习动态诊断策略，优化交互效果和最终准确性；（iii）DiagBench，一个多中心诊断基准，用于评估多轮诊断交互轨迹。该基准包含2200个由医生验证的病例，来源于4种不同的分布，以及3300个由医生撰写的评分细则用于细粒度的过程导向评估。（iv）广泛的评估显示，DiagAgent在域内和域外（OOD）设置中均表现出色。DiagAgent在11个最先进的LLM和2个提示工程代理中显著领先。在端到端设置中，其诊断准确率提高了11.20%，检查推荐F1分数提高了17.58%，并且在所有三个外部中心中均保持了最先进的性能。此外，在基于评分细则的评估中，它在加权评分分数上超过了次优模型7.1%。这些发现表明，在交互临床环境中学习策略能够赋予长期的诊断管理能力，而这种能力无法通过被动训练实现。
英文摘要：
We present a framework for training large language models (LLMs) as diagnostic agents with reinforcement learning, enabling them to manage multi-turn interactive diagnostic processes, adaptively select examinations, and commit to final diagnoses. Unlike instruction-tuned models trained on static data, our method acquires diagnostic strategies through dynamic exploration and outcome-based feedback, mapping evolving patient states to the next optimal examination and subsequent diagnosis. Our contributions include: (i) DiagGym, a diagnostics world model trained with electronic health records, serving as a virtual clinical environment to support closed-loop in-silico training and evaluation for interactive diagnosis; (ii) DiagAgent, trained via end-to-end multi-turn RL to learn dynamic diagnostic policies that optimize both interactive effectiveness and final accuracy; (iii) DiagBench, a multi-center diagnostic benchmark designed to evaluate multi-turn diagnostic interaction trajectories. The benchmark comprises 2.2K physician-validated cases sourced from 4 distinct distributions, alongside 3.3K physician-written rubrics for granular process-oriented evaluation. (iv) Extensive evaluations demonstrate DiagAgent's superior performance across both in-domain and out-of-domain (OOD) settings. DiagAgent significantly outperforms 11 SOTA LLMs and 2 prompt-engineered agents. In the end-to-end setting, it delivers a 11.20% increase in diagnostic accuracy and a 17.58% boost in examination recommendation F1 score, while consistently maintaining SOTA performance across all three external centers. Furthermore, in rubric-based evaluations, it surpasses the next-best model by 7.1% in weighted rubric score. These findings indicate that learning policies in interactive clinical environments confers long-term diagnostic management abilities unattainable through passive training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
