发表机构
NVIDIA（英伟达）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对医疗联邦学习中客户端存在缺失模态的问题，提出FedCoRe框架，通过学习表示或对数空间修正，在呼吸恶化任务中恢复了ECG和CXR缺失导致的部分性能损失。
AI 中文摘要
联邦多模态模型通常假设每个站点都拥有所有模态数据，但医院在电子健康记录（EHR）、胸部X光片（CXR）和心电图（ECG）的获取渠道上存在差异。我们基于MIMIC衍生的呼吸恶化任务，采用模拟的联邦学习（FL）客户端来研究该场景，并提出FedCoRe（联邦跨模态表示补全）。FedCoRe学习表示空间或对数空间的修正，而非生成合成ECG或CXR图像。当客户端观察到某一模态且该模态可能在部署时缺失时，它会通过使用和不使用该模态评估同一示例以获取配对监督信号。仅拥有此类配对数据的客户端会更新补全模块，而验证过程可保留未改变的预测。我们在评估期间冻结训练好的多模态预测器，以确保测量到的差异仅来自补全过程。隐藏ECG会使AUROC降低约0.085；采用配对示例的FedAvg方法可恢复0.0415的AUROC，即损失性能的49.0%。因此，我们报告两种不同的效果：采用配对示例的FedAvg可部分恢复缺失ECG带来的性能差距，而通过验证选择的补全是针对特定任务的分类器对数修正，而非字面意义上的ECG恢复。对于CXR，在隐藏CXR的受控测试中，感知效果的补全可恢复52.8%的损失。采用配对示例的FedAvg可传递部分此类效果，但在部署时输入缺少CXR的情况下，验证会保留无补全的基线。因此，FedCoRe可理解为一个验证门控的补全/修正框架：它可在支持的场景中恢复缺失模态的信号，但仅当配对示例和验证证据支持该模态时才可部署。
英文摘要
Federated multimodal models often assume every site has every modality, although hospitals differ in access to EHRs, chest radiographs, and ECGs. We study this setting on a MIMIC-derived respiratory deterioration task with simulated FL clients and introduce FedCoRe (Federated Cross-Modal Representation Completion). FedCoRe learns representation- or logit-space corrections rather than generating synthetic ECGs or CXR images. When a client observes a modality that may be missing at deployment, it evaluates the same example with and without that modality to obtain paired supervision. Only clients with such pairs update the completion module, and validation may retain the unchanged prediction. We freeze the trained multimodal predictor during evaluation so that measured differences come only from completion. Hiding ECG reduced AUROC by about 0.085; paired-example FedAvg restored 0.0415 AUROC, or 49.0% of the lost performance. We therefore report two distinct effects: paired-example FedAvg partially recovers the missing-ECG gap, while validation-selected completion is a task-specific classifier-logit correction rather than literal ECG recovery. For CXR, effect-aware completion recovers 52.8% of the loss in a controlled test where CXR is hidden. Paired-example FedAvg transfers part of this effect, but validation keeps the no-completion baseline for deployment cases whose inputs lack CXR. Thus, FedCoRe should be read as a validation-gated completion/correction framework: it can recover missing-modality signal in supported settings, but it should be deployed only when paired examples and validation evidence support that modality.
CommentsAccepted to the 7th Workshop on Distributed, Collaborative & Federated Learning, DeCaF 2026, MICCAI, Strasbourg, France
