arXiv 2604.107552026-05-13cs.CV

MMRareBench: 一种罕见疾病多模态和多图像医学基准

MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmark

Shanghai AI Laboratory（上海人工智能实验室）；Imperial College London（帝国理工学院）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）

Junzhi Ning, Jiashi Lin, Yingying Fang, Wei Li, Jiyao Liu, Cheng Tang, Chenglong Ma, Wenhao Tang, Tianbin Li, Ziyan Huang, Guang Yang, Junjun He

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出MMRareBench，首个评估多模态和多图像临床能力的罕见疾病基准，包含1756个问题-答案对和7958张医学图像，揭示23个MLLMs在多图像任务中表现低下，存在能力稀释效应。

摘要：

多模态大语言模型（MLLMs）在常见疾病临床任务中取得了进展，但其在罕见疾病中的表现仍缺乏系统评估。在罕见疾病场景中，医生通常缺乏先验临床知识，被迫依赖病例层面的证据进行临床判断。现有基准主要评估常见疾病、单图像设置，导致多模态和多图像证据整合在罕见疾病数据稀缺情况下未被系统评估。我们引入MMRareBench，据我们所知，这是首个联合评估多模态和多图像临床能力的罕见疾病基准，涵盖四个流程对齐的轨道：诊断、治疗规划、跨图像证据对齐和检查建议。该基准包含1756个问题-答案对和7958张关联医学图像，均来自PMC病例报告，具有Orphanet锚定的本体对齐、轨道特定的泄漏控制、证据驱动的标注和两级评估协议。对23个MLLMs的系统评估揭示了能力碎片化特征和普遍低治疗规划性能，尽管在诊断得分上具有竞争力，医疗领域模型在多图像轨道上仍显著落后于通用MLLMs。这些模式与能力稀释效应一致：医疗微调可以缩小诊断差距，但可能削弱罕见疾病证据整合所需的组成多图像能力。

英文摘要：

Multimodal large language models (MLLMs) have advanced clinical tasks for common conditions, but their performance on rare diseases remains largely untested. In rare-disease scenarios, clinicians often lack prior clinical knowledge, forcing them to rely strictly on case-level evidence for clinical judgments. Existing benchmarks predominantly evaluate common-condition, single-image settings, leaving multimodal and multi-image evidence integration under rare-disease data scarcity systematically unevaluated. We introduce MMRareBench, to our knowledge the first rare-disease benchmark jointly evaluating multimodal and multi-image clinical capability across four workflow-aligned tracks: diagnosis, treatment planning, cross-image evidence alignment, and examination suggestion. The benchmark comprises 1,756 question-answer pairs with 7,958 associated medical images curated from PMC case reports, with Orphanet-anchored ontology alignment, track-specific leakage control, evidence-grounded annotations, and a two-level evaluation protocol. A systematic evaluation of 23 MLLMs reveals fragmented capability profiles and universally low treatment-planning performance, with medical-domain models trailing general-purpose MLLMs substantially on multi-image tracks despite competitive diagnostic scores. These patterns are consistent with a capacity dilution effect: medical fine-tuning can narrow the diagnostic gap but may erode the compositional multi-image capability that rare-disease evidence integration demands.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。