arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.23629cs.CV
收藏

DDX-TRACE: 视觉语言模型中医学诊断轨迹的基准

DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs

Technical University of Munich(慕尼黑技术大学);TUM University Hospital(TUM大学医院);Munich Center for Machine Learning(慕尼黑机器学习中心);LMU Munich(慕尼黑大学);Aalto University(阿尔托大学);Imperial College London(伦敦帝国学院)

Jiazhen Pan, Weixiang Shen, Jun Li, Julian Canisius, Felix Bitzer, Paula Roßmüller, Jiancheng Yang, Virginie Kreutzinger, Daniel Rueckert, Benedikt Wiestler

AI总结:

提出DDX-TRACE基准,通过隐藏证据的多轮诊断轨迹评估VLM在神经放射学中的工作流质量,揭示最终诊断分数无法反映的推理缺陷。

摘要:

医学诊断并非来自完全指定的病例的单次预测。它是一个序贯工作流程:临床医生决定获取哪些证据,修订鉴别诊断,并在诊断得到充分支持时停止。大多数医学AI基准则提前揭示相关背景,仅对最终答案评分,使得无依据的正确猜测、过早闭合、低效工作流以及不良的不确定性更新变得不可见。我们引入了DDX-TRACE,一个由医生裁决的多模态神经放射学基准,在211个具有挑战性的病例中评估隐藏证据下的诊断轨迹。每个病例从有限的临床病史开始;模型以自由形式请求影像研究,在可用时接收匹配的图像包,每轮后更新概率性鉴别诊断,并以定位的最终诊断结束。评估最先进的VLM,我们发现最终诊断分数可能严重歪曲工作流质量:模型可能在没有必要证据的情况下猜测合理的诊断,请求有用的研究但误解原始图像,或者低效地获取证据同时更新不确定性不佳。受控证据变体隔离了规划、视觉证据提取和下游鉴别推理中的瓶颈。DDX-TRACE将医学AI评估从最终答案转向证据支持的诊断轨迹。

英文摘要:

Medical diagnosis is not a single prediction from a fully specified vignette. It is a sequential workup: clinicians decide what evidence to obtain, revise a differential diagnosis, and stop when the diagnosis is sufficiently supported. Most medical AI benchmarks instead reveal the relevant context upfront and score only the final answer, making unsupported correct guesses, premature closure, inefficient workups, and poor uncertainty updating invisible. We introduce DDX-TRACE, a physician-adjudicated benchmark for multimodal neuroradiology that evaluates diagnostic trajectories under hidden evidence over 211 challenging cases. Each case begins with limited clinical history; models request imaging studies in free form, receive matched image bundles when available, update a probabilistic differential diagnosis after each turn, and stop with a localized final diagnosis. Evaluating state-of-the-art VLMs, we find that final diagnosis scores can substantially misrepresent workup quality: models may guess plausible diagnoses without essential evidence, request useful studies but misinterpret raw images, or acquire evidence inefficiently while updating uncertainty poorly. Controlled evidence variants isolate bottlenecks in planning, visual evidence extraction, and downstream differential reasoning. DDX-TRACE shifts medical AI evaluation from final answers to evidence-supported diagnostic trajectories.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑