发表机构
Weill Cornell Medicine（威尔康奈尔医学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出基于来源的框架用于构建渐进式多模态诊断对话，并评估MLLMs的诊断推理能力，实验显示所提框架转换病例报告的参考对话表现优异，而前沿MLLMs的相关指标显著更低。
AI 中文摘要
临床诊断需要逐步整合患者病史、体格检查、实验室检查结果、医学影像及诊断相关检查。然而，大多数多模态医学基准评估的是固定输入或最终答案，而完全交互式诊断智能体将证据选择与证据解释混为一谈。我们提出了一个基于来源的框架，用于从病例报告构建渐进式多模态诊断对话，以及一种评估策略，用于评估多模态大语言模型（MLLMs）在最终诊断、诊断推理及影像发现解释方面的表现。对24份内科病例报告的评估显示，我们的框架能够准确将病例报告转换为参考对话，诊断F1值达0.99，推理质量评分（满分5分）为4.79。对两款前沿MLLMs（o4-mini和Claude Haiku 4.5）的评估显示，它们的推理质量评分分别为2.75和2.50，诊断、推理及影像发现的F1值均显著更低。结果表明，流畅的响应不一定反映基于证据的临床推理，凸显了所提框架在评估多模态诊断推理方面的实用性。
英文摘要
Clinical diagnosis requires progressive integration of patient history, physical examination, laboratory findings, medical images, and diagnostic-informative tests. However, most multimodal medical benchmarks evaluate fixed inputs or endpoint answers, while fully interactive diagnostic agents conflate evidence selection with evidence interpretation. We present a source-grounded framework to construct progressive multimodal diagnostic dialogues from case reports and an evaluation strategy for assessing MLLMs on final diagnosis, diagnostic reasoning, and image-finding interpretation. Evaluation on 24 internal medicine case reports showed that our framework can accurately convert case reports into reference dialogues, achieving a diagnosis F1 of 0.99 and a reasoning-quality score of 4.79 out of 5. Evaluation on two frontier MLLMs (o4-mini and Claude Haiku 4.5) achieved reasoning-quality scores of 2.75 and 2.50, respectively, with substantially lower diagnosis, reasoning, and image-finding F1 scores. The results demonstrate that fluent responses do not necessarily reflect evidence-grounded clinical reasoning and highlight the utility of the proposed framework for evaluating multimodal diagnostic reasoning.
CommentsAccepted to IEEE HealthCom 2026, Distinguished Invited Papers Track
