评估大语言模型在罕见病诊断中的表现：一项基于《豪斯医生》的案例研究
Evaluating Large Language Models on Rare Disease Diagnosis: A Case Study using House M.D
AI总结：
本文构建基于《豪斯医生》的176组症状-诊断配对罕见病诊断数据集，评估四款主流大语言模型的叙事医疗推理能力，建立相关基线与公开评估框架以支撑AI辅助诊断研究。
AI中文摘要：
大语言模型（LLMs）已在多个领域展现出相应能力，但其根据叙事性医疗病例开展罕见病诊断的表现尚未得到充分探究。我们引入了一个全新数据集，包含从《豪斯医生》中提取的176组症状-诊断配对数据，这部医疗题材电视剧已被验证可用于医学教育中的罕见病识别教学。我们在基于叙事的诊断推理任务上评估了GPT 4o mini、GPT 5 mini、Gemini 2.5 Flash、Gemini 2.5 Pro四款当前最优的大语言模型。结果显示各模型表现差异显著，准确率区间为16.48%至38.64%，新一代模型的性能较前代提升了2.3倍。尽管所有模型在罕见病诊断上都面临相当大的挑战，但不同架构模型观测到的性能提升指明了颇具前景的未来发展方向。我们这个经教育场景验证的基准为叙事性医疗推理建立了基线性能指标，同时提供了公开可用的评估框架，可用于推进AI辅助诊断相关研究。
英文摘要：
Large language models (LLMs) have demonstrated capabilities across diverse domains, yet their performance on rare disease diagnosis from narrative medical cases remains underexplored. We introduce a novel dataset of 176 symptom-diagnosis pairs extracted from House M.D., a medical television series validated for teaching rare disease recognition in medical education. We evaluate four state-of-the-art LLMs such as GPT 4o mini, GPT 5 mini, Gemini 2.5 Flash, and Gemini 2.5 Pro on narrative-based diagnostic reasoning tasks. Results show significant variation in performance, ranging from 16.48% to 38.64% accuracy, with newer model generations demonstrating a 2.3 times improvement. While all models face substantial challenges with rare disease diagnosis, the observed improvement across architectures suggests promising directions for future development. Our educationally validated benchmark establishes baseline performance metrics for narrative medical reasoning and provides a publicly accessible evaluation framework for advancing AI-assisted diagnosis research.
