MIMIC-RD: 能否在真实临床环境中让大语言模型对罕见病进行差异性诊断？
MIMIC-RD: Can LLMs differentially diagnose rare diseases in real-world clinical settings?
- University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
- University of Illinois College of Medicine（伊利诺伊大学医学学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
MIMIC-RD通过直接映射临床文本实体到Orphanet，评估LLM在真实临床环境下的罕见病差异性诊断能力，发现现有模型表现不佳，揭示了临床需求与现有能力之间的差距。
AI中文摘要：
尽管罕见病影响着10%的美国人，但其差异性诊断仍然具有挑战性。由于其出色的回忆能力，大型语言模型（LLMs）最近被探索用于差异性诊断。现有评估基于LLM的罕见病诊断方法存在两个关键限制：它们依赖于理想化的临床案例研究，无法捕捉真实世界临床复杂性，或者它们使用ICD代码作为疾病标签，这显著低估了罕见病，因为许多疾病缺乏直接映射到如Orphanet这样的全面罕见病数据库。为了解决这些限制，我们探索了MIMIC-RD，一个通过直接将临床文本实体映射到Orphanet构建的罕见病差异性诊断基准。我们的方法涉及一个初始的LLM挖掘过程，随后由四位医学标注员进行验证，以确认识别出的实体确实是真正的罕见病。我们在145名患者的数据集上评估了各种模型，发现当前最先进的LLM在罕见病差异性诊断上表现不佳，突显了现有能力与临床需求之间的巨大差距。从我们的发现中，我们概述了若干未来步骤，以改进罕见病的差异性诊断。
英文摘要：
Despite rare diseases affecting 1 in 10 Americans, their differential diagnosis remains challenging. Due to their impressive recall abilities, large language models (LLMs) have been recently explored for differential diagnosis. Existing approaches to evaluating LLM-based rare disease diagnosis suffer from two critical limitations: they rely on idealized clinical case studies that fail to capture real-world clinical complexity, or they use ICD codes as disease labels, which significantly undercounts rare diseases since many lack direct mappings to comprehensive rare disease databases like Orphanet. To address these limitations, we explore MIMIC-RD, a rare disease differential diagnosis benchmark constructed by directly mapping clinical text entities to Orphanet. Our methodology involved an initial LLM-based mining process followed by validation from four medical annotators to confirm identified entities were genuine rare diseases. We evaluated various models on our dataset of 145 patients and found that current state-of-the-art LLMs perform poorly on rare disease differential diagnosis, highlighting the substantial gap between existing capabilities and clinical needs. From our findings, we outline several future steps towards improving differential diagnosis of rare diseases.
