MOSAIC：面向客户端特定缺失模态的联邦图像级弱监督肿瘤分割的模态无关频谱对齐方法

中文摘要 AI 辅助

临床环境中可靠的多模态融合需处理各机构间不完整且异构的模态子集，而隐私约束禁止集中式数据共享。联邦学习（FL）缓解了数据共享约束，但面临客户端特定缺失模态问题——各机构拥有不完整的多模态子集，这会降低融合质量和分割性能。尽管联邦学习与弱监督已分别被研究，但结合图像级标签处理异构缺失模态的场景仍未解决。我们提出MOSAIC，首个面向客户端特定缺失模态的模态无关联邦框架，用于弱监督二值肿瘤分割。我们引入客户端特定模态对齐模块，在无需模态身份先验知识的情况下将可用通道融合到共享潜在空间；提出频谱原型对齐损失，利用紧凑非可逆频域统计量协调跨客户端分布偏移；还设计专用联邦优化网络，将生成的类激活图（CAM）伪标签去噪为精确掩码，突破弱监督的精度上限。在三个多机构脑肿瘤基准（FeTS2022、BraTS-MEN和BraTS-SSA）上的实验表明，该方法在所有图像、框和点监督基线中均取得显著提升，仅用图像级标签就接近全监督精度，在FeTS2022上达到0.84的Dice分数。新增动态客户端加入功能，使未见过的机构能在0.01-0.04 Dice的精度损失内加入已训练的联邦网络，无需重新训练。代码可在指定URL获取。

英文摘要

Trustworthy multimodal fusion in clinical settings requires handling incomplete and heterogeneous modality subsets across institutions, where privacy constraints prohibit centralized data sharing. Federated learning (FL) mitigates data-sharing constraints but suffers from client-specific missing modalities, where institutions possess incomplete multimodal subsets, degrading fusion quality and segmentation performance. While FL and weak supervision have been studied separately, their joint use with image-level labels under heterogeneous missing modalities remains unaddressed. We propose \textbf{MOSAIC}, the first modality-agnostic federated framework for weakly supervised binary tumor segmentation under client-specific missing modalities. We introduce a client-specific modality-alignment module that fuses available channels into a shared latent space without prior knowledge of modality identity, a spectral prototype alignment loss that reconciles cross-client distribution shift using compact non-invertible frequency-domain statistics, and a dedicated federated refinement network that denoises the resulting CAM pseudo-labels into accurate masks, breaking the accuracy ceiling of weak supervision. Experiments on three multi-institutional brain tumor benchmarks (FeTS2022, BraTS-MEN, and BraTS-SSA) demonstrate significant improvements over all image, box, and point-supervised baselines, approaching fully supervised accuracy using only image-level labels and reaching 0.84 Dice on FeTS2022. Dynamic new client addition enables previously unseen institutions to join an already-trained federation within 0.01-0.04 Dice without retraining. Code is available at https://github.com/Tarun2201/MOSAIC.

发表机构

• Indian Institute of Science（印度科学学院）

机构由 AI 辅助整理，请以论文原文为准。