arXiv 2512.234402025-12-29cs.CL

ClinDEF: 一种用于大语言模型临床推理的动态评估框架

ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning

ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University（浙江大学杭州全球科技创新中心）；ZJU-UIUC Institute, Zhejiang University（浙江大学UIUC研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The Polytechnic Institute, Zhejiang University（浙江大学 polytechnic 院）；AntGroup（蚂蚁集团）

Yuqi Tang, Jing Yu, Zichang Su, Kehua Feng, Zhihui Zhu, Libin Wang, Lei Liang, Qiang Zhang, Keyan Ding, Huajun Chen

备注：: 23 pages, 4 figures, under review

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ClinDEF通过模拟诊断对话评估大语言模型的临床推理能力，利用疾病知识图谱生成病例并进行多轮交互，提供细粒度效率分析和诊断质量评估，揭示LLM在临床推理中的关键不足。

摘要：

临床诊断始于医生与患者之间的互动，在此过程中，医生迭代地收集信息，通过患者的反应确定检查并细化鉴别诊断。这一动态的临床推理过程目前由现有LLM基准测试所缺乏，这些测试专注于静态的问题回答。为了弥补这些差距，最近的方法探索了包含交互式临床对话的动态医疗框架。尽管有效，但它们往往依赖于有限且易受污染的数据集，并缺乏细粒度、多层次的评估。在本工作中，我们提出了ClinDEF，一种通过模拟诊断对话来评估大语言模型临床推理的动态框架。基于疾病知识图谱，我们的方法动态生成患者病例，并促进基于LLM的医生与自动患者代理之间的多轮互动。我们的评估协议超越了诊断准确性，纳入了细粒度效率分析和基于评分表的诊断质量评估。实验显示，ClinDEF有效揭示了最先进的LLM中的关键临床推理差距，提供了一种更细致且具有临床意义的评估范式。

英文摘要：

Clinical diagnosis begins with doctor-patient interaction, during which physicians iteratively gather information, determine examination and refine differential diagnosis through patients' response. This dynamic clinical-reasoning process is poorly represented by existing LLM benchmarks that focus on static question-answering. To mitigate these gaps, recent methods explore dynamic medical frameworks involving interactive clinical dialogues. Although effective, they often rely on limited, contamination-prone datasets and lack granular, multi-level evaluation. In this work, we propose ClinDEF, a dynamic framework for assessing clinical reasoning in LLMs through simulated diagnostic dialogues. Grounded in a disease knowledge graph, our method dynamically generates patient cases and facilitates multi-turn interactions between an LLM-based doctor and an automated patient agent. Our evaluation protocol goes beyond diagnostic accuracy by incorporating fine-grained efficiency analysis and rubric-based assessment of diagnostic quality. Experiments show that ClinDEF effectively exposes critical clinical reasoning gaps in state-of-the-art LLMs, offering a more nuanced and clinically meaningful evaluation paradigm.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。