In this study, we propose a novel federated learning (FL) approach that utilizes 3D style transfer for the multi-organ segmentation task. The multi-organ dataset, obtained by integrating multiple datasets, has high scalability and can improve generalization performance as the data volume increases. However, the heterogeneity of data owing to different clients with diverse imaging conditions and target organs can lead to severe overfitting of local models. To align models that overfit to different local datasets, existing methods require frequent communication with the central server, resulting in higher communication costs and risk of privacy leakage. To achieve an efficient and safe FL, we propose an Anatomical 3D Frequency Domain Generalization (A3DFDG) method for FL. A3DFDG utilizes structural information of human organs and clusters the 3D styles based on the location of organs. By mixing styles based on these clusters, it preserves the anatomical information and leads models to learn intra-organ diversity, while aligning the optimization of each local model. Experiments indicate that our method can maintain its accuracy even in cases where the communication cost is highly limited (=1.25% of the original cost) while achieving a significant difference compared to baselines, with a higher global dice similarity coefficient score of 4.3%. Despite its simplicity and minimal computational overhead, these results demonstrate that our method has high practicality in real-world scenarios where low communication costs and a simple pipeline are required. The code used in this project will be publicly available.
Domain generalization (DG) has been a hot topic in image recognition, with a goal to train a general model that can perform well on unseen domains. Recently, federated learning (FL), an emerging machine learning paradigm to train a global model from multiple decentralized clients without compromising data privacy, brings new challenges, also new possibilities, to DG. In the FL scenario, many existing state-of-the-art (SOTA) DG methods become ineffective, because they require the centralization of data from different domains during training. In this paper, we propose a novel domain generalizati…
JP Morgan Chase（摩根大通）；University of Cambridge（剑桥大学）
Chest Computational Tomography (CT) scans present low cost, speed and objectivity for COVID-19 diagnosis and deep learning methods have shown great promise in assisting the analysis and interpretation of these images. Most hospitals or countries can train their own models using in-house data, however empirical evidence shows that those models perform poorly when tested on new unseen cases, surfacing the need for coordinated global collaboration. Due to privacy regulations, medical data sharing between hospitals and nations is extremely difficult. We propose a GAN-augmented federated learning m…
Eindhoven University of Technology（埃因霍温理工大学）；Philips Research Laboratories（飞利浦研究院）；Philips Healthcare（飞利浦医疗）
We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De-normalization (SPADE) technique for conditional image synthesis to preserve the semantic spatial information of ground truth anatomy. Using the parameterized motion model of the XCAT heart, we genera…
Imperial College London（伦敦帝国学院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希·亚历山大大学）
This paper presents the use of Multi-Agent Reinforcement Learning (MARL) to perform navigation in 3D anatomical volumes from medical imaging. We utilize Neural Style Transfer to create synthetic Computed Tomography (CT) agent gym environments and assess the generalization capabilities of our agents to clinical CT volumes. Our framework does not require any labelled clinical data and integrates easily with several image translation techniques, enabling cross modality applications. Further, we solely condition our agents on 2D slices, breaking grounds for 3D guidance in much more difficult imagi…
Multiscale Structure-Guided Latent Diffusion for Multimodal MRI Translation
Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程系，沈阳，中国）；Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China（教育部医学图像智能计算重点实验室，东北大学，沈阳，中国）；National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, China（工业智能与系统优化国家级前沿科学中心，沈阳，中国）；Amii, University of Alberta, Edmonton, Alberta, Canada（阿尔伯塔大学阿米人工智能研究所，埃德蒙顿，阿尔伯塔，加拿大）；AiShiWeiLai AI Research, Beijing, China（人工智能研究（北京）有限公司，北京，中国）
Stony Brook University（石溪大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；UT Southwestern Medical Center（德克萨斯大学西南医学中心）
Deep learning has shown great promise for CT image reconstruction, in particular to enable low dose imaging and integrated diagnostics. These merits, however, stand at great odds with the low availability of diverse image data which are needed to train these neural networks. We propose to overcome this bottleneck via a deep reinforcement learning (DRL) approach that is integrated with a style-transfer (ST) methodology, where the DRL generates the anatomical shapes and the ST synthesizes the texture detail. We show that our method bears high promise for generating novel and anatomically accurat…
EndoLive: Real-Time Style Transfer for Endoscopic Endonasal Skull Base Surgical Video
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh School of Computing and Information（匹兹堡大学计算与信息学院）；School of Computing and Information（计算与信息学院）
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）
Various deep learning models have been developed to segment anatomical structures from medical images, but they typically have poor performance when tested on another target domain with different data distribution. Recently, unsupervised domain adaptation methods have been proposed to alleviate this so-called domain shift issue, but most of them are designed for scenarios with relatively small domain shifts and are likely to fail when encountering a large domain gap. In this paper, we propose DCDA, a novel cross-modality unsupervised domain adaptation framework for tasks with large domain shif…
Biomedical Image Analysis Group, Imperial College London（伦敦帝国理工学院生物医学图像分析小组）；Data Science Institute, Imperial College London（伦敦帝国理工学院数据科学研究所）；Department of Medicine, Imperial College London（伦敦帝国理工学院医学院）
In this work, we present a fully automatic method to segment cardiac structures from late-gadolinium enhanced (LGE) images without using labelled LGE data for training, but instead by transferring the anatomical knowledge and features learned on annotated balanced steady-state free precession (bSSFP) images, which are easier to acquire. Our framework mainly consists of two neural networks: a multi-modal image translation network for style transfer and a cascaded segmentation network for image segmentation. The multi-modal image translation network generates realistic and diverse synthetic LGE …
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳人工智能与机器人学会）；Institute of High Performance Computing (IHPC), A*STAR（高性能计算研究所 (IHPC)，A*STAR）
This work studies training generative adversarial networks under the federated learning setting. Generative adversarial networks (GANs) have achieved advancement in various real-world applications, such as image editing, style transfer, scene generations, etc. However, like other deep learning models, GANs are also suffering from data limitation problems in real cases. To boost the performance of GANs in target tasks, collecting images as many as possible from different sources becomes not only important but also essential. For example, to build a robust and accurate bio-metric verification sy…
Task-Adaptive 3D Cross-Field MRI Translation via Field-Conditioned Content-Style Pretraining
School of Integrated Circuits and Electronics, Beijing Institute of Technology（北京理工大学集成电路与电子学院）；School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University（清华大学医学院生物医学工程学院）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）
Decouple, Purify and Unite: Semantic-Structural Prototype Learning for Federated Medical Segmentation
Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（北京协和医院，中国医学科学院北京协和医学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Nanyang Technological University（南洋理工大学）；Ant Group（蚂蚁集团）；Shanghai Innovation Institute（上海创新研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Fourth Military Medical University（中国人民解放军空军军医大学）；Macao Polytechnic University（澳门理工大学）；Wuhan University（武汉大学）；The Hong Kong Polytechnic University（香港理工大学）
Recent research has made great progress in realizing neural style transfer of images, which denotes transforming an image to a desired style. Many users start to use their mobile phones to record their daily life, and then edit and share the captured images and videos with other users. However, directly applying existing style transfer approaches on videos, i.e., transferring the style of a video frame by frame, requires an extremely large amount of computation resources. It is still technically unaffordable to perform style transfer of videos on mobile phones. To address this challenge, we pr…
University of Padova（帕多瓦大学）；Politecnico di Torino（都灵理工大学）
Federated Learning (FL) has recently emerged as a possible way to tackle the domain shift in real-world Semantic Segmentation (SS) without compromising the private nature of the collected data. However, most of the existing works on FL unrealistically assume labeled data in the remote clients. Here we propose a novel task (FFREEDA) in which the clients' data is unlabeled and the server accesses a source labeled dataset for pre-training only. To solve FFREEDA, we propose LADD, which leverages the knowledge of the pre-trained model by employing self-supervision with ad-hoc regularization techniq…
Deep Anatomical Federated Network (Dafne): An open client-server framework for the continuous, collaborative improvement of deep learning-based medical image segmentation
University of Basel（巴塞尔大学）；University Hospital Basel（巴塞尔大学医院）；University of Pavia（帕维亚大学）；Leiden University Medical Center（莱顿大学医学中心）；Siemens Healthineers International AG（西门子医疗国际股份公司）；Balgrist Campus（巴尔格里斯特校区）；Peking Union Medical College Hospital（北京协和医院）；Istituti Clinici Scientifici Maugeri IRCCS（莫杰里科学临床研究所）；IRCCS Mondino Foundation（蒙迪诺基金会国立卫生研究院）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Stanford University（斯坦福大学）
CO-EVO: Co-evolving Semantic Anchoring and Style Diversification for Federated DG-ReID
School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件学院）；QiYuan Lab（启元实验室）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；School of Software, Tsinghua University（清华大学软件学院）
MLA-BIN: Model-level Attention and Batch-instance Style Normalization for Domain Generalization of Federated Learning on Medical Image Segmentation
School of Computer and Communication Engineering, Zhengzhou University of Light Industry（郑州轻工业大学计算机与通信工程学院）；Department of Applied Computing, Michigan Technological University（密歇根理工大学应用计算系）
TU Darmstadt（达姆施塔特工业大学）；University Hospital Cologne（科隆大学医院）
Although deep federated learning has received much attention in recent years, progress has been made mainly in the context of natural images and barely for computational pathology. However, deep federated learning is an opportunity to create datasets that reflect the data diversity of many laboratories. Further, the effort of dataset construction can be divided among many. Unfortunately, existing algorithms cannot be easily applied to computational pathology since previous work presupposes that data distributions of laboratories must be similar. This is an unlikely assumption, mainly since dif…
Harmonization Across Imaging Locations(HAIL): One-Shot Learning for Brain MRI
Children’s National Hospital（美国儿童国家医疗中心）；George Washington University（乔治·华盛顿大学）；Children’s Hospital Colorado（科罗拉多儿童医院）；Children’s Hospital of Philadelphia（费城儿童医院）
PathFL: Multi-Alignment Federated Learning for Pathology Image Segmentation
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（新型一代人工智能技术及其交叉应用重点实验室（东南大学））；Department of Biostatistics, Center for Global Health, School of Public Health, Nanjing Medical University（公共卫生学院生物统计学系，全球健康中心，南京医科大学）；Institute of High-Performance Computing, Agency for Science, Technology and Research（高性能计算研究所，科技研究局）
Xiamen University（厦门大学）；University College London（伦敦大学学院）
Medical image segmentation under federated learning (FL) is a promising direction by allowing multiple clinical sites to collaboratively learn a global model without centralizing datasets. However, using a single model to adapt to various data distributions from different sites is extremely challenging. Personalized FL tackles this issue by only utilizing partial model parameters shared from global server, while keeping the rest to adapt to its own data distribution in the local training of each site. However, most existing methods concentrate on the partial parameter splitting, while do not c…
Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Model
College of Computer Science, Sichuan University（四川大学计算机学院）；Key Laboratory of Data Protection and Intelligent Management, Ministry of Education, Sichuan University（四川大学（教育部数据保护与智能管理重点实验室））；School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）；School of Computer Science and Software Engineering, Southwest Petroleum University（西南石油大学计算机科学与技术学院）；Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）
Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill（放射科与生物医学研究成像中心（BRIC）、北卡罗来纳大学教堂山分校）；Lampe Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University（Lampe联合生物医学工程部门、北卡罗来纳大学教堂山分校和北卡罗来纳州立大学）
SegHeD+: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints and Lesion-aware Augmentation
Imperial College London（帝国理工学院）；Data Science Institute, Imperial College London（帝国理工学院数据科学研究所）；UK Dementia Research Institute, Imperial College London（帝国理工学院英国痴呆症研究所）；Rosalind Franklin Institute（罗莎琳德·富兰克林研究所）
LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learning
Department of Computer Science and Engineering, Department of Chemical and Biological Engineering and Division of Life Science, Hong Kong University of Science and Technology（计算机科学与工程系、化学与生物工程系及生命科学分校，香港科学与技术大学）
SegHeD: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints
Imperial College London（伦敦帝国学院）；Data Science Institute, Imperial College London（伦敦帝国学院数据科学研究所）；Beijing Institute of Technology（北京理工大学）；UK Dementia Research Institute（英国痴呆症研究所）；Rosalind Franklin Institute（罗莎琳德·富兰克林研究所）
Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imaging
College of Computer Science, Sichuan University（四川大学计算机学院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）；Department of Electrical and Computer Engineering, The University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）
Deep Learning-based Assessment of the Relation Between the Third Molar and Mandibular Canal on Panoramic Radiographs using Local, Centralized, and Federated Learning
PULSE: A Unified Multi-Task Architecture for Cardiac Segmentation, Diagnosis, and Few-Shot Cross-Modality Clinical Adaptation
King Fahd University of Petroleum；Institute for Medical Engineering \& Science, Massachusetts Institute of Technology, US；Harvard Medical School, Harvard University, US；KFUPM–SDAIA Joint Research Centre for Artificial Intelligence, Saudi Arabia
Federated Domain Generalization via Prompt Learning and Aggregation
School of Computing and Artificial Intelligence, Shandong University of Finance and Economics（山东财经大学计算与人工智能学院）；School of Computer Science and Technology, Shandong Jianzhu University（山东建筑大学计算机科学与技术学院）；College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）
本文提出 PLAN 框架，通过提示学习与聚合在联邦域泛化场景中适配视觉-语言模型，利用本地和全局提示的安全交换与聚合提升未见域泛化能力，同时显著降低计算与通信开销。
LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3
University College London（伦敦大学学院）；University of Cambridge（剑桥大学）；Cambridge Institute of Therapeutic Immunology and Infectious Diseases（剑桥治疗免疫学与传染病研究所）；Wellcome Trust Sanger Institute（惠康基金会桑格研究所）；Cambridge Stem Cell Institute（剑桥干细胞研究所）；Centre for AI in Medicine（医学人工智能中心）；Cavendish Laboratory（卡文迪什实验室）；Medical University of Vienna（维也纳医科大学）；European Synchrotron Radiation Facility（欧洲同步辐射装置）；CIFAR（加拿大高级研究所）；Cambridge University Hospitals NHS Foundation Trust（剑桥大学医院NHS基金会信托基金）；NIHR Cambridge Biomedical Research Centre（英国国家健康与研究院剑桥生物医学研究中心）
