在多模态医学视觉基础模型中学习涌现的模块化表示
Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models
AI总结：
本文提出Director-Experts (DEX)框架，通过调控模块化动态，在多模态医学视觉基础模型中学习稳定的模块化表示，并在新的医学视觉基准数据集上验证了其在26个下游任务中的优越性。
摘要：
多模态医学视觉（MV）基础模型（FM）在异质成像模态间面临显著的非独立同分布（Non-IID）特征统计挑战。对这类数据进行单一监督优化会引发冲突梯度，导致表示向模态主导的捷径坍缩。本文将这一失败重新解释为涌现模块化中专门化与协调之间的失衡，并提出Director-Experts（DEX）模块化网络，该网络在堆叠模块中显式调控这些动态。每个DEX模块包含一组专家，通过我们的图像级激活策略动态适应，自主专注于模态主导的统计特征，同时结合通过我们组指数移动平均更新的Director，将多专家知识蒸馏到共享空间，实现跨模态的语义整合，从而驱动模块化表示的涌现。我们构建了一个新的基准数据集Medical Vision Universe，包含超过400万张图像，覆盖10种模态，为DEX提供了最广泛的模态覆盖的FM级预训练。在26个下游任务上的广泛评估表明，DEX在优化行为和迁移性方面有所改进，表明DEX是通用多模态医学AI的有原则的一步。我们的代码和数据集将在https://github.com/YutingHe-list/DEX上公开。
英文摘要：
Multi-modality medical vision (MV) foundation models (FM) are fundamentally challenged by pronounced Non-IID feature statistics across heterogeneous imaging modalities. Monolithic self-supervised optimization on such data induces conflicting gradients, driving representations to collapse toward modality-dominant shortcuts. This work reframes this failure as an imbalance between specialization and coordination in emergent modularity, and proposes Director-Experts (DEX), a modular network that explicitly regulates these dynamics in stacked modules. Each DEX module comprises a pool of experts, dynamically adapted by our image-wise activation strategy, autonomously specializing in modality-dominant statistics, together with a director, updated via our group exponential moving average, which distills multi-expert knowledge into a shared space for semantic integration across modalities, thus driving the emergence of modular representations. We curate a new benchmark, Medical Vision Universe, over 4 million images across 10 modalities, which provides a FM-level pre-training with the broadest coverage of distinct imaging modalities to our DEX. Extensive evaluations on 26 downstream tasks demonstrate improved optimization behavior and transferability, indicating DEX as a principled step toward general-purpose multi-modality medical AI. Our code and dataset will be opened at https://github.com/YutingHe-list/DEX.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
