发表机构
Center for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对医学通才模型性能不足的问题，提出Med-RADIO多教师蒸馏框架，将多个领域专家知识压缩至统一视觉基础模型，在五种模态分类基准上提升通才模型性能并接近专家水平。
AI 中文摘要
大规模医学数据集和计算资源的快速扩展推动了医学基础模型的显著进步。鉴于医学影像模态固有的异质性，当前研究主要遵循两条路径：针对特定模态优化的专用模型，以及设计用于处理多种模态的通才模型。然而，医学通才模型既面临相对于自然图像通才模型而言训练数据规模不足的问题，也面临相对于医学专家模型而言领域特定深度不足的问题。经验上，通才模型建立了跨模态性能基线，而专家模型则在其各自领域内定义了性能上限。为了将这些基线提升至这些上限，我们提出了Med-RADIO，一个医学多教师蒸馏框架，通过将多个领域特定教师的互补专业知识压缩到一个统一的医学视觉基础模型中，将所有领域归为一体。我们的方法精选通才和专家教师，分配模态对齐的蒸馏流以重组通才预训练数据，使其与专家领域匹配，并使用平衡损失以防止任何单一教师主导蒸馏过程。在涵盖五种模态的内部和外部分类基准上，Med-RADIO在线性探测下优于强大的医学通才模型，并在大多数评估模态上与代表性专家模型保持竞争力。代码可在以下网址获取：https://this https URL。
英文摘要
The rapid expansion of large-scale medical datasets and computational resources has driven significant progress in medical foundation models. Given the inherent heterogeneity of medical imaging modalities, current research mainly follows two paths: specialized models optimized for specific modalities, and generalist models designed to handle multiple modalities. However, medical generalist models suffer from both insufficient training data scale relative to natural image generalists and inadequate domain-specific depth relative to medical specialists. Empirically, generalist models establish a cross-modality performance baseline, while specialists define the performance ceiling within their respective domains. To elevate this baseline toward these ceilings, we propose Med-RADIO, a medical multi-teacher distillation framework that Reduces All Domains Into One by compressing complementary expertise from multiple domain-specific teachers into a unified medical vision foundation model. Our method curates both generalist and specialist teachers, allocates modality-aligned distillation streams to reorganize generalist pretraining data so it matches specialist domains, and uses a balanced loss to prevent any single teacher from dominating the distillation process. On internal and external classification benchmarks spanning five modalities, Med-RADIO improves over strong medical generalists under linear probing and remains competitive with representative specialists on most evaluated modalities. Code is available at https://github.com/CAIR-HKISI/Med-RADIO.
