CoM³eT：通过联邦多维上下文集成实现医学图像分析的基础模型
CoM$^3$eT: A foundation model for medical image analysis through federated, multidimensional context integration
浏览论文内容
中文总结 AI 辅助
CoM³eT是统一多专科、多预测类型及多维度输入的医学视觉基础模型，在公开竞赛中表现优于同类模型，仅微调少量参数即可适配多临床任务，联邦学习场景下性能接近聚合数据训练。
中文摘要 AI 辅助
医学基础模型在有限标注数据下训练AI模型时可提升泛化能力，但仍局限于单一专科（如病理学或放射学），且仅支持稀疏或密集输出（如分类或分割）。本文提出CoM³eT（Co-representation Multidimensional Multitask Medical Transformer，即共表征多维多任务医学Transformer），这一医学视觉基础模型通过注意力机制建模多维上下文，统一了病理学与放射学、稀疏与密集预测、二维及更高维输入。CoM³eT在包含5个断层扫描、4个全标本、3个二维数据集的公开竞赛中表现优于其他医学基础模型，覆盖稀疏与密集预测任务及报告生成。在适配不同临床应用时，仅训练不足2.5%的参数即可达到与全微调相当的性能，使无需高性能GPU集群的研究成为可能；应用于跨医院联邦学习时，该方法在互联网连接及消费级硬件上的性能可与聚合数据训练相当。
英文摘要
Medical foundation models improve generalization when training AI models with limited labeled data, but remain confined to a single specialty, such as pathology or radiology, and to either sparse or dense outputs, such as classification or segmentation. Here, we present CoM$^3$eT (Co-representation Multidimensional Multitask Medical Transformer), a medical vision foundation model that unifies pathology and radiology, sparse and dense predictions, and two- and higher-dimensional inputs by modeling multidimensional context with attention. CoM$^3$eT outperformed other medical foundation models in an open competition spanning five tomographic, four whole-specimen, and three two-dimensional datasets, covering sparse and dense prediction tasks as well as report generation. When adapted across diverse clinical applications, training fewer than 2.5% of parameters achieved performance comparable to full fine-tuning, enabling research without access to high-performance GPU clusters. Applied to federated learning across hospitals, this approach achieved performance comparable to pooled-data training over internet connections and with consumer-grade hardware.
发表机构
- Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
- RWTH Aachen University（亚琛工业大学）
- Medizinische Hochschule Hannover（汉诺威医学院）
- Massachusetts General Hospital（麻省总医院）
- Harvard Medical School（哈佛医学院）
- Charité – Universitätsmedizin Berlin（柏林夏里特医学院）
- Freie Universität Berlin（柏林自由大学）
- Humboldt-Universität zu Berlin（柏林洪堡大学）
机构由 AI 辅助整理，请以论文原文为准。
