DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs
Technical University of Munich(慕尼黑技术大学);TUM University Hospital(TUM大学医院);Munich Center for Machine Learning(慕尼黑机器学习中心);LMU Munich(慕尼黑大学);Aalto University(阿尔托大学);Imperial College London(伦敦帝国学院)
Jiazhen Pan, Weixiang Shen, Jun Li, Julian Canisius, Felix Bitzer, Paula Roßmüller, Jiancheng Yang, Virginie Kreutzinger, Daniel Rueckert, Benedikt Wiestler
Medical diagnosis is not a single prediction from a fully specified vignette. It is a sequential workup: clinicians decide what evidence to obtain, revise a differential diagnosis, and stop when the diagnosis is sufficiently supported. Most medical AI benchmarks instead reveal the relevant context upfront and score only the final answer, making unsupported correct guesses, premature closure, inefficient workups, and poor uncertainty updating invisible. We introduce DDX-TRACE, a physician-adjudicated benchmark for multimodal neuroradiology that evaluates diagnostic trajectories under hidden evidence over 211 challenging cases. Each case begins with limited clinical history; models request imaging studies in free form, receive matched image bundles when available, update a probabilistic differential diagnosis after each turn, and stop with a localized final diagnosis. Evaluating state-of-the-art VLMs, we find that final diagnosis scores can substantially misrepresent workup quality: models may guess plausible diagnoses without essential evidence, request useful studies but misinterpret raw images, or acquire evidence inefficiently while updating uncertainty poorly. Controlled evidence variants isolate bottlenecks in planning, visual evidence extraction, and downstream differential reasoning. DDX-TRACE shifts medical AI evaluation from final answers to evidence-supported diagnostic trajectories.
Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design
Department of Computer Science and Software Engineering (CSSE) Concordia University(计算机科学与软件工程系(CSSE)康科迪亚大学);University of Montreal Hospital Center(蒙特利尔大学医院中心);Mila–Quebec AI Institute(魁北克人工智能研究所)
Evidence-based diagnostic reasoning with multi-agent copilot for human pathology
Health Sciences and Technology, Harvard-MIT(哈佛-MIT健康科学与技术);Department of Pathology, Massachusetts General Hospital, Harvard Medical School(麻省总医院病理科,哈佛医学院);Cancer Program, Broad Institute of Harvard and MIT(哈佛-MIT博德研究所癌症项目);Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院);Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT)(麻省理工学院电气工程与计算机科学);Harvard Data Science Initiative, Harvard University(哈佛大学数据科学计划)
MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage
Mohamed bin Zayed University of Artificial Intelligence, UAE(马尔代夫布扎伊德人工智能大学,阿联酋);National Institute of Technology, Silchar(西尔CHAR国家理工学院);Birmingham City University, UK(伯明翰城市大学,英国)
3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks
ZJU-Angelalign R&D Center for Intelligence Healthcare(浙江大学智能医疗研发中心);Zhejiang University(浙江大学);Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence(浙江省医学影像人工智能重点实验室);Guangdong Institute of Intelligence Science and Technology(广东省智能科学与技术研究院)
Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models
AI4LIFE, Hanoi University of Science and Technology, Vietnam(AI4LIFE,越南科学与技术大学);SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, France(SAMOVAR,法国电信南巴黎学院,巴黎理工学院);Military Central Hospital, Vietnam(越南108军中心医院)
Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations
University of Tehran(德黑兰大学);Tehran University of Medical Sciences(德黑兰医科大学);Advanced Diagnostic and Interventional Radiology Research Center (ADIR)(高级诊断与介入放射学研究中心(ADIR))
Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);Eisai Inc.(卫材株式会社);IISc, Bangalore(印度科学研究所班加罗尔分校);Cohere Labs Community(Cohere实验室社区)
LeafNet: A Large-Scale Dataset and Comprehensive Benchmark for Foundational Vision-Language Understanding of Plant Diseases
School of Electrical Engineering, Korea University(韩国大学电气工程学院);Department of Integrative Biology, The University of Texas at Austin(德克萨斯大学奥斯汀分校整合生物学系);Department of Software Engineering, FPT University(FPT大学软件工程系)
Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究院,合肥国家科学中心);Anhui Polytechnic University(安徽理工大学);Hefei University of Technology(合肥工业大学);Anhui University(安徽大学);IGS, Imperial College London(帝国理工学院伦敦分校)
Contextualized Visual Personalization in Vision-Language Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea(电气电子工程系,首尔国立大学,首尔,韩国);Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea(人工智能交叉学科项目,首尔国立大学,首尔,韩国)
Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification
College of Computer Science, Sichuan University(四川大学计算机科学学院);Yong Loo Lin School of Medicine, National University of Singapore(新加坡国立大学杨潞龄医学院);Key Laboratory of Data Protection and Intelligent Management, Ministry of Education, Sichuan University(四川大学数据保护与智能管理教育部重点实验室);National Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology(北京理工大学自主智能无人系统国家重点实验室);Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局高性能计算研究所)