FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models
China Academy of Electronics and Information Technology（中国电子科技集团公司电子科学研究院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）
Remote sensing data is often distributed across multiple institutions, and due to privacy concerns and data-sharing restrictions, leveraging large-scale datasets in a centralized training framework is challenging. Federated learning offers a promising solution by enabling collaborative model training across distributed data sources without requiring data centralization. However, current Vision-Language Models (VLMs), which typically contain billions of parameters, pose significant communication challenges for traditional federated learning approaches based on model parameter updates, as they would incur substantial communication costs. In this paper, we propose FedRSCLIP, the first federated learning framework designed for remote sensing image classification based on a VLM, specifically CLIP. FedRSCLIP addresses the challenges of data heterogeneity and large-scale model transmission in federated environments by introducing Prompt Learning, which optimizes only a small set of tunable parameters. The framework introduces a dual-prompt mechanism, comprising Shared Prompts for global knowledge sharing and Private Prompts for client-specific adaptation. To maintain semantic coherence between shared and private prompts, we propose the Dual Prompt Alignment Constraint to balance global consistency and local adaptability across diverse client distributions. Additionally, to enhance cross-modal representation learning, we introduce the Cross-Modal Feature Alignment Constraint to align multimodal features between text and image prompts. To validate the effectiveness of our proposed model, we construct a Fed-RSIC dataset based on three existing remote sensing image classification datasets, specifically designed to simulate various federated learning configurations. Experimental results demonstrate the effectiveness and superiority of FedRSCLIP in remote sensing image classification.
On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing
BIFOLD - Berlin Institute for the Foundations of Learning and Data（BIFOLD - 柏林学习与数据基础研究所）；Technische Universität Berlin（柏林工业大学）；National Technical University of Athens（雅典国家技术大学）；National Observatory of Athens（雅典国家天文台）；Harokopio University of Athens（哈罗科皮奥雅典大学）
Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning for Collaborative Remote Sensing Semantic Segmentation
School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）；Space Research Institute, Russian Academy of Sciences（俄罗斯科学院空间研究所）；College of Engineering, Purdue University（普渡大学工程学院）；Institute of Surface-Earth System Science, School of Earth System Science, Tianjin University（天津大学地球系统科学学院地表-地球系统科学研究所）
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）；Fudan University（复旦大学）
The growing need for technology that supports remote healthcare is being acutely highlighted by an aging population and the COVID-19 pandemic. In health-related machine learning applications the ability to learn predictive models without data leaving a private device is attractive, especially when these data might contain features (e.g., photographs or videos of the body) that make identifying a subject trivial and/or the training data volume is large (e.g., uncompressed video). Camera-based remote physiological sensing facilitates scalable and low-cost measurement, but is a prime example of a…
FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
College of Computing and Informatics, Sungkyunkwan University（顺天大学计算与信息学院）；Department of Computer Science and Engineering, University of Michigan（密歇根大学计算机科学与工程系）；Konkuk Aerospace Design-Airworthiness Institute, Konkuk University（konkuk大学航空航天设计-适航研究所）；Department of Electronic Systems, Aalborg University（奥尔堡大学电子系统系）；Department of Information and Communication Engineering, Noakhali Science and Technology University（诺阿克利科学与技术大学信息与通信工程系）
A Multi-Modal Federated Learning Framework for Remote Sensing Image Classification
Technische Universität Berlin（柏林工业大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classification
Technische Universität Berlin（柏林工业大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
FedX: Explanation-Guided Pruning for Communication-Efficient Federated Learning in Remote Sensing
Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学学院，柏林技术大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）
Recent advancements in deep learning methods bring computer-assistance a step closer to fulfilling promises of safer surgical procedures. However, the generalizability of such methods is often dependent on training on diverse datasets from multiple medical institutions, which is a restrictive requirement considering the sensitive nature of medical data. Recently proposed collaborative learning methods such as Federated Learning (FL) allow for training on remote datasets without the need to explicitly share data. Even so, data annotation still represents a bottleneck, particularly in medicine a…
University of Padova（帕多瓦大学）；Politecnico di Torino（都灵理工大学）
Federated Learning (FL) has recently emerged as a possible way to tackle the domain shift in real-world Semantic Segmentation (SS) without compromising the private nature of the collected data. However, most of the existing works on FL unrealistically assume labeled data in the remote clients. Here we propose a novel task (FFREEDA) in which the clients' data is unlabeled and the server accesses a source labeled dataset for pre-training only. To solve FFREEDA, we propose LADD, which leverages the knowledge of the pre-trained model by employing self-supervision with ad-hoc regularization techniq…
University of Pittsburgh（匹兹堡大学）；University of Notre Dame（圣母大学）
Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can help in this regard by learning a shared model while keeping training data local for privacy. Traditional FL requires fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain due to high labeling cost and the requirement of expertise. Contrastive learning (CL), as a self-supervised learning approach, can effectivel…
AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
Institute of Advanced Research in Artificial Intelligence（人工智能高等研究院）；Nanyang Technological University（南洋理工大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Lancaster University（兰卡斯特大学）；University of Southampton（南安普顿大学）
Noise-Aware and Dynamically Adaptive Federated Defense Framework for SAR Image Target Recognition
Shanxi Key Laboratory of Cryptography and Data Security, School of Computer Science and Artificial Intelligence, Shanxi Normal University（山西密码与数据安全重点实验室，计算机科学与人工智能学院，山西师范大学）；School of Computer Science and Technology, Guizhou University（贵州大学计算机科学与技术学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；School of Electrical Engineering, Guangxi University（广西大学电气工程学院）；School of Computing, Engineering and Mathematical Science, La Trobe University（拉筹伯大学计算、工程与数学科学学院）；College of Cyber Security, Jinan University（济南大学网络安全学院）
University of Pittsburgh（匹兹堡大学）；University of Notre Dame（圣母大学）；George Mason University（乔治梅森大学）
Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can learn a shared model from decentralized data. But traditional FL requires fully-labeled data for training, which is very expensive to obtain. Self-supervised contrastive learning (CL) can learn from unlabeled data for pre-training, followed by fine-tuning with limited annotations. However, when adopting CL in FL, the limited data diversity on eac…
Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania（生物医学影像计算与分析中心）；Department of Radiology, Perelman School of Medicine, University of Pennsylvania（放射科）；Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania（病理学与实验室医学部）；Department of Informatics, Technical University of Munich（信息学院，慕尼黑技术大学）；Division of Medical Image Computing, German Cancer Research Center (DKFZ)（医学影像计算部，德国癌症研究中心）；Intel Corporation（英特尔公司）；Division of Computer Assisted Medical Interventions, DKFZ（计算机辅助医疗干预部，德国癌症研究中心）；Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania（生物统计学、流行病学与信息学部，佩尔曼医学院，宾夕法尼亚大学）；Department of Quantitative Biomedicine, University of Zurich（定量生物医学部，苏黎世大学）；Translational Image-guided Oncology, Institute for AI in Medicine (IKIM), University Hospital Essen（转化影像引导肿瘤学，医学人工智能研究所（IKIM），埃森大学医院）；Leidos Biomedical Research, Inc., Frederick National Laboratory for Cancer Research（莱迪奥斯生物医学研究公司，弗雷德里克国家癌症研究中心）；Cancer Imaging Program, National Cancer Institute, National Institutes of Health（癌症成像计划，国家癌症研究所，国家卫生研究院）；Department of Diagnostic Radiology, University of Texas MD Anderson Cancer Center（诊断放射科，德克萨斯MD安德森癌症中心）；Neuroimaging Informatics and Analysis Center, Washington University（神经影像信息与分析中心，华盛顿大学）；Department of Radiology, Washington University（放射科，华盛顿大学）；Department of Neurology, The University of Alabama at Birmingham（神经科，阿拉巴马大学伯明翰分校）；Institute for Surgical Technology and Biomechanics, University of Bern（手术技术与生物力学研究所，伯尔尼大学）；Support Centre for Advanced Neuroimaging Inselspital, Institute for Diagnostic and Interventional Neuroradiology,Bern University Hospital（高级神经影像支持中心，诊断与介入神经放射学研究所，伯尔尼大学医院）；Center for MR-Research, University Children’s Hospital Zurich（磁共振研究中心，苏黎世儿童医院）；Tata Memorial Centre, Homi Bhabha National Institute（塔塔纪念中心，霍米·巴布哈国家研究所）；Hillman Cancer Center, University of Pittsburgh Medical Center（希尔曼癌症中心，匹兹堡医学中心）
