A Step Toward Federated Pretraining of Multimodal Large Language Models
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统实验室）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；Pengcheng Laboratory（鹏城实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）
The rapid evolution of Multimodal Large Language Models (MLLMs) is bottlenecked by the saturation of high-quality public data, while vast amounts of diverse multimodal data remain inaccessible in privacy-sensitive silos. Federated Learning (FL) offers a promising solution to unlock these distributed resources, but existing research focuses predominantly on fine-tuning, leaving the foundational pre-training phase largely unexplored. In this paper, we formally introduce the Federated MLLM Alignment (Fed-MA) task, a lightweight pre-training paradigm that freezes the vision encoder and LLM while collaboratively training the cross-modal projector. We identify two critical challenges in this setting: (i) parameter interference in aggregating local projectors; and (ii) gradient oscillations in one-pass collaborative SGD. To address these challenges, we propose Fed-CMP, a pioneering framework for federated MLLM pre-training. Fed-CMP employs Canonical Reliability-Aware Aggregation, which constructs a canonical space to decompose client projectors into a shared alignment basis and client-specific coefficients, then performs reliability-weighted fusion to suppress parameter interference. Furthermore, Fed-CMP introduces Orthogonality-Preserved Momentum, which applies momentum to the shared alignment basis via orthogonal projection, accumulating historical optimization directions while preserving geometric structure. We construct four federated pre-training scenarios based on public datasets, and extensive experiments validate that Fed-CMP significantly outperforms existing baselines.
FedUAF: Uncertainty-Aware Fusion with Reliability-Guided Aggregation for Multimodal Federated Sentiment Analysis
Shanghai University（上海大学）；North China Electric Power University（华北电力大学）；Université de Technologie de Compiègne（法国图卢兹国立理工学院）；Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；Macquarie University（麦考瑞大学）
Federated Learning in Big Model Era: Domain-Specific Multimodal Large Models
ENN Group Co.,Ltd.（ENN集团有限公司）；China University of Petroleum（中国石油大学）；Unicom (Shanghai) Industrial Internet Co.,Ltd.（联通（上海）工业互联网有限公司）；Qingdao Windaka Technology Co.,Ltd.（青岛Windaka科技有限公司）
Multimodal Federated Learning in Healthcare: a Review
West Virginia University（西弗吉尼亚大学）；Tribhuvan University（特里布万大学）；University at Albany（奥尔巴尼大学）；University of Pittsburgh（匹兹堡大学）；University of Aberdeen（阿伯丁大学）
BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Models
School of Information and Communication Engineering, Dalian University of Technology（信息与通信工程学院，大连理工大学）；School of Computer Science and Technology, Anhui University（计算机科学与技术学院，安徽大学）；New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences (CASIA)（模式识别新实验室（NLPR）多模态人工智能系统国家重点实验室（MAIS）自动化研究所，中国科学院（CASIA））
Multimodal Federated Learning via Contrastive Representation Ensemble
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Waterloo（滑铁卢大学）；Carnegie Mellon University（卡内基梅隆大学）
Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections
National Center for Tumor Diseases (NCT)（国家肿瘤中心）；Translational Surgical Oncology（转化外科肿瘤学）；Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology（医学系和图林根大学技术大学卡尔·古斯塔夫·卡尔医院）；DKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology（德累斯顿大学技术大学医学系和德累斯顿癌症研究中心）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗斯多夫亥姆霍兹中心）；Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden（具有人类在环的触觉互联网中心，德累斯顿技术大学）；Visceral, Thoracic and Vascular Surgery, Faculty of Medicine and University Hospital Carl Gustav Carus, TU Dresden（visceral、胸外科和血管外科，医学系和图林根大学技术大学卡尔·古斯塔夫·卡尔医院）；Weldon School of Biomedical Engineering, Purdue University（生物医学工程韦尔登学校，普渡大学）；Medical Oncology, NCT, University Hospital Heidelberg, Germany（医学肿瘤学，国家肿瘤中心，海德堡大学医院，德国）；Medicine I, Faculty of Medicine and University Hospital Carl Gustav Carus, TU Dresden（医学I，医学系和图林根大学技术大学卡尔·古斯塔夫·卡尔医院）
Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Washington State University（华盛顿州立大学）；VinUniversity（文大学）；Griffin University（格里芬大学）；Hanoi University of Science and Technology（河内科学技术大学）
Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregation
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanjing University（南京大学）；University of California, Irvine（加州大学欧文分校）；The Chinese University of Hong Kong（香港中文大学）
Pilot: Building the Federated Multimodal Instruction Tuning Framework
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Pengcheng Laboratory（鹏城实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
LMU Munich（慕尼黑莱布尼茨大学）；Technical University of Munich（慕尼黑技术大学）；Siemens Technology（西门子技术）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University Heidelberg（海德堡大学）
FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Data
Nanjing University of Science and Technology（南京理工大学）；National University of Singapore（新加坡国立大学）；Institute of High-Performance Computing, A*STAR（新加坡高性能计算研究院（新加坡科研局下属机构））
Department of Biomedical Engineering, Linköping University, Sweden（_linköping大学生物医学工程系）；Department of Computer and Information Science, Linköping University, Sweden（_linköping大学计算机与信息科学系）；Center for Medical Image Science and Visualization (CMIV), Linköping University, Sweden（_linköping大学医学影像科学与可视化中心）
Explore the Limits of Omni-modal Pretraining at Scale
The Chinese University of Hong Kong（香港中文大学）；University of Chinese Academy of Sciences（中国科学院大学）；Chinese Academy of Science（中国科学院）；Shanghai AI Laboratory（上海人工智能实验室）
FedAFD: Multimodal Federated Learning via Adversarial Fusion and Distillation
Zhejiang Key Laboratory of Space Information Sensing and Transmission, Hangzhou Dianzi University（浙江空间信息感知与传输重点实验室，杭州电子大学）；Laboratory of Complex Systems Modeling and Simulation, School of Computer Science and Technology, Hangzhou Dianzi University（复杂系统建模与仿真实验室，计算机科学与技术学院，杭州电子大学）；Alibaba International Digital Commerce Group（阿里巴巴国际数字商业集团）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
PID-Guided Partial Alignment for Multimodal Decentralized Federated Learning
Department of Electrical and Computer Engineering, Stony Brook University（石溪大学电气与计算机工程系）；Department of Applied Mathematics and Statistics and the Department of Computer Science, Stony Brook University（石溪大学应用数学与统计系和计算机科学系）；Department of Electrical and Computer Engineering, New York University（纽约大学电气与计算机工程系）；Department of Computer Science, New York Institute of Technology（纽约理工学院计算机科学系）
Multimodal Federated Learning With Missing Modalities through Feature Imputation Network
University of Aberdeen（阿伯丁大学）；NepAl Applied Mathematics and Informatics Institute for research（尼泊尔应用数学与信息技术研究所）；West Virginia University（西弗吉尼亚大学）；Fogsphere (Redev.AI)（Fogsphere（Redev.AI））
Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality
Kyung Hee University（庆熙大学）；The University of Danang—Vietnam-Korea University of Information and Communication Technology（岘港大学—越韩信息与通信技术大学）；Digital Science and Technology Institute（数字科学技术研究院）
MQFL-FHE: Multimodal Quantum Federated Learning Framework with Fully Homomorphic Encryption
SVKM's Dwarkadas J. Sanghvi College of Engineering（SVKM的达沃拉德·J·桑格维工程学院）；eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD)（eBRAIN实验室，工程系，纽约大学阿布扎比分校（NYUAD））；Center for Quantum and Topological Systems (CQTS), NYUAD Research Institute（量子与拓扑系统中心（CQTS），NYUAD研究机构）；The Maersk Mc-Kinney Moller Institute, University of Southern Denmark（马士克·麦克金尼·莫勒研究所，南丹麦大学）；Corvinus Institute for Advanced Studies (CIAS), Budapest, Hungary（科维努斯高级研究学院（CIAS），布达佩斯，匈牙利）；Davidson School of Chemical Engineering, Purdue University（戴维森化学工程学院，普渡大学）
FedMultiEmo: Real-Time Emotion Recognition via Multimodal Federated Learning
Institute of Industrial Automation and Software Engineering, University of Stuttgart, Germany（工业自动化与软件工程研究所，斯图加特大学）；Electrical Engineering, University of Stuttgart, Germany（电气工程，斯图加特大学）；Graduate School of Excellence advanced Manufacturing Engineering, University of Stuttgart, Germany（先进制造工程优秀研究生院，斯图加特大学）
