注意罕见病：罕见皮肤疾病能否通过诊断推理可靠诊断？
Mind the Rarities: Can Rare Skin Diseases Be Reliably Diagnosed via Diagnostic Reasoning?
- Carnegie Mellon University（卡内基梅隆大学）
- University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
- Australian National University（澳大利亚国立大学）
- University of Western Australia（西澳大学）
- POSTECH
- Yale University（耶鲁大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出DermCase基准，通过多模态数据评估罕见皮肤疾病诊断推理能力，揭示现有模型在诊断准确性和临床推理中的不足。
AI中文摘要：
大型视觉-语言模型（LVLMs）在皮肤病学中表现出色；然而，评估罕见病症的诊断推理仍鲜有研究。现有基准聚焦常见疾病，仅评估最终准确性，忽视临床推理过程，这对复杂病例至关重要。我们通过构建DermCase长上下文基准，基于同行评审病例报告，包含26,030个多模态图像-文本对和6,354个临床挑战性病例，每个病例均标注了全面的临床信息和逐步推理链。为实现可靠评估，我们建立了基于DermLIP的相似性度量，能更契合皮肤科医生评估鉴别诊断质量。对22个领先LVLMs的基准测试揭示了诊断准确性、鉴别诊断和临床推理方面的显著缺陷。微调实验显示，指令微调显著提升性能，而直接偏好优化（DPO）收益有限。系统性错误分析进一步揭示了当前模型推理能力的关键限制。
英文摘要：
Large vision-language models (LVLMs) demonstrate strong performance in dermatology; however, evaluating diagnostic reasoning for rare conditions remains largely unexplored. Existing benchmarks focus on common diseases and assess only final accuracy, overlooking the clinical reasoning process, which is critical for complex cases. We address this gap by constructing DermCase, a long-context benchmark derived from peer-reviewed case reports. Our dataset contains 26,030 multi-modal image-text pairs and 6,354 clinically challenging cases, each annotated with comprehensive clinical information and step-by-step reasoning chains. To enable reliable evaluation, we establish DermLIP-based similarity metrics that achieve stronger alignment with dermatologists for assessing differential diagnosis quality. Benchmarking 22 leading LVLMs exposes significant deficiencies across diagnosis accuracy, differential diagnosis, and clinical reasoning. Fine-tuning experiments demonstrate that instruction tuning substantially improves performance while Direct Preference Optimization (DPO) yields minimal gains. Systematic error analysis further reveals critical limitations in current models' reasoning capabilities.
