arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14160cs.AI
收藏

过程监督多智能体强化学习用于可靠的临床推理

Process-Supervised Multi-Agent Reinforcement Learning for Reliable Clinical Reasoning

School of Informatics, University of Edinburgh, UK(信息学院,爱丁堡大学)

Chaeeun Lee, T. Michael Yates, Pasquale Minervini, T. Ian Simpson

AI总结:

本文提出了一种过程监督的多智能体强化学习框架,用于提升基因-疾病有效性校准的临床推理准确性与过程忠实度。

摘要:

临床决策需要在异质性证据上进行细致推理并提供可追溯的论证。尽管最近的LLM多智能体系统(MAS)显示出潜力,但它们主要优化结果准确性,而忽略了与临床标准一致的过程基础推理。一个关键的现实案例是基因-疾病有效性校准,专家必须通过整合多样化的生物医学证据来确定基因是否在疾病中起因果作用。我们为此任务引入了一种智能体作为工具的强化学习框架,具有两个目标:(i)过程层面的监督以确保推理遵循有效的临床路径,以及(ii)通过分层多智能体系统实现高效的协调。我们在ClinGen数据集上的评估表明,使用仅结果奖励时,使用GRPO训练的Qwen3-4B监督智能体将最终结果准确性从基础模型监督的0.195提高到0.732,但导致过程对齐性较差(0.392 F1)。相反,使用过程+结果奖励时,使用GRPO训练的监督智能体实现了更高的结果准确性(0.750),同时显著提高了过程的忠实度到0.520 F1。我们的代码可在https://github.com/chaeeunlee-io/GeneDiseaseCurationAgents上获得。

英文摘要:

Clinical decision-making requires nuanced reasoning over heterogeneous evidence and traceable justifications. While recent LLM multi-agent systems (MAS) show promise, they largely optimise for outcome accuracy while overlooking process-grounded reasoning aligned with clinical standards. One critical real-world case of this is gene-disease validity curation, where experts must determine whether a gene is causally implicated in a disease by synthesising diverse biomedical evidence. We introduce an agent-as-tool reinforcement learning framework for this task with two objectives: (i) process-level supervision to ensure reasoning follows valid clinical pathways, and (ii) efficient coordination via a hierarchical multi-agent system. Our evaluation on the ClinGen dataset shows that with outcome-only rewards, MAS with a GRPO-trained Qwen3-4B supervisor agent substantially improves final outcome accuracy from 0.195 with a base model supervisor to 0.732, but results in poor process alignment (0.392 F1). Conversely, with process + outcome rewards, MAS with GRPO-trained supervisor achieves higher outcome accuracy (0.750) while significantly improving process fidelity to 0.520 F1. Our code is available at https://github.com/chaeeunlee-io/GeneDiseaseCurationAgents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑