arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.10039cs.AI
收藏

MSDiagnosis:评估大语言模型在多步临床诊断中的基准

MSDiagnosis: A Benchmark for Evaluating Large Language Models in Multi-Step Clinical Diagnosis

Ruijin Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属瑞金医院)

Ruihui Hou, Shencheng Chen, Yongqi Fan, Guangya Yu, Lifeng Zhu, Jing Sun, Jingping Liu, Tong Ruan

AI总结:

针对现有临床诊断任务多为单步、不符合实际多步流程的问题,本文提出中文基准MSDiagnosis(含12科室2,225病例),并设计结合前向/后向推理、反思与细化的框架,使大语言模型能自我调整诊断,实验验证了其有效性。

摘要:

临床诊断在医疗实践中至关重要,通常需要一个连续且不断发展的过程,包括初步诊断、鉴别诊断和最终诊断。然而,现有的大多数临床诊断任务是单步过程,这与真实临床环境中复杂的多步诊断流程不符。在本文中,我们提出了一个中文临床诊断基准,称为MSDiagnosis。该基准包含来自12个科室的2,225个病例,涵盖初步诊断、鉴别诊断和最终诊断等任务。此外,我们提出了一种新颖且有效的框架。该框架结合了前向推理、后向推理、反思和细化,使大语言模型能够自我评估并调整其诊断结果。为此,我们测试了开源模型、闭源模型以及我们提出的框架。实验结果表明了所提出方法的有效性。我们还提供了全面的实验分析,并为该任务提出了未来的研究方向。

英文摘要:

Clinical diagnosis is critical in medical practice, typically requiring a continuous and evolving process that includes primary diagnosis, differential diagnosis, and final diagnosis. However, most existing clinical diagnostic tasks are single-step processes, which does not align with the complex multi-step diagnostic procedures found in real-world clinical settings. In this paper, we propose a Chinese clinical diagnostic benchmark, called MSDiagnosis. This benchmark consists of 2,225 cases from 12 departments, covering tasks such as primary diagnosis, differential diagnosis, and final diagnosis. Additionally, we propose a novel and effective framework. This framework combines forward inference, backward inference, reflection, and refinement, enabling the large language model to self-evaluate and adjust its diagnostic results. To this end, we test open-source models, closed-source models, and our proposed framework.The experimental results demonstrate the effectiveness of the proposed method. We also provide a comprehensive experimental analysis and suggest future research directions for this task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑