Federated Learning (FL) enables learning a shared model across many clients without violating the privacy requirements. One of the key attributes in FL is the heterogeneity that exists in both resource and data due to the differences in computation and communication capacity, as well as the quantity and content of data among different clients. We conduct a case study to show that heterogeneity in resource and data has a significant impact on training time and model accuracy in conventional FL systems. To this end, we propose TiFL, a Tier-based Federated Learning System, which divides clients into tiers based on their training performance and selects clients from the same tier in each training round to mitigate the straggler problem caused by heterogeneity in resource and data quantity. To further tame the heterogeneity caused by non-IID (Independent and Identical Distribution) data and resources, TiFL employs an adaptive tier selection approach to update the tiering on-the-fly based on the observed training performance and accuracy overtime. We prototype TiFL in a FL testbed following Google's FL architecture and evaluate it using popular benchmarks and the state-of-the-art FL benchmark LEAF. Experimental evaluation shows that TiFL outperforms the conventional FL in various heterogeneous conditions. With the proposed adaptive tier selection policy, we demonstrate that TiFL achieves much faster training performance while keeping the same (and in some cases - better) test accuracy across the board.
In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, {we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum}. We …
George Mason University（乔治梅森大学）；IBM Research - Almaden（IBM阿尔马登研究院）；Emory University（埃默里大学）
Federated learning (FL) involves training a model over massive distributed devices, while keeping the training data localized. This form of collaborative learning exposes new tradeoffs among model convergence speed, model accuracy, balance across clients, and communication cost, with new challenges including: (1) straggler problem, where the clients lag due to data or (computing and network) resource heterogeneity, and (2) communication bottleneck, where a large number of clients communicate their local updates to a central server and bottleneck the server. Many existing FL methods focus on op…
We consider decentralized model training in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. To reduce communication overhead, the clients in each silo perform multiple local gradient steps before sharing updates with their hub. Each hub adjusts its…
School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）；Department of Computer Science and Engineering, Yonsei University（延世大学计算机科学与工程系）
Purdue University（普渡大学）；University at Buffalo (SUNY)（纽约州立大学布法罗分校）；Duke University（杜克大学）
We investigate training machine learning (ML) models across a set of geo-distributed, resource-constrained clusters of devices through unmanned aerial vehicles (UAV) swarms. The presence of time-varying data heterogeneity and computational resource inadequacy among device clusters motivate four key parts of our methodology: (i) stratified UAV swarms of leader, worker, and coordinator UAVs, (ii) hierarchical nested personalized federated learning (HN-PFL), a distributed ML framework for personalized model training across the worker-leader-core network hierarchy, (iii) cooperative UAV resource p…
Hierarchical Federated Learning for Networked AI: From Communication Saving to Architecture-Aware Design
Department of Electrical Engineering, Chalmers University of Technology（查尔姆斯理工大学电子工程系）；Centre for Wireless Communications, University of Oulu（奥卢大学无线通信中心）；Department of Electrical Engineering, Yale University（耶鲁大学电子工程系）
Design of Two-Level Incentive Mechanisms for Hierarchical Federated Learning
Nanjing University of Science and Technology（南京理工大学）；Hainan University（海南大学）；The Hong Kong Polytechnic University（香港理工大学）；East China Normal University（华东师范大学）；Shanghai Jiao Tong University（上海交通大学）
Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；School of Engineering, Westlake University（西湖大学工学院）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）
Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomeration
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Laboratory（中关村实验室）；Tsinghua University（清华大学）；Beijing Jiaotong University（北京交通大学）
Agglomerative Federated Learning: Empowering Larger Model Training via End-Edge-Cloud Collaboration
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Laboratory（中关村实验室）；Beijing Jiaotong University（北京交通大学）
The University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Federated learning (FL) enables collaborative model training without centralizing data. However, the traditional FL framework is cloud-based and suffers from high communication latency. On the other hand, the edge-based FL framework that relies on an edge server co-located with mobile base station for model aggregation has low communication latency but suffers from degraded model accuracy due to the limited coverage of edge server. In light of high accuracy but high-latency cloud-based FL and low-latency but low-accuracy edge-based FL, this paper proposes a new FL framework based on cooperativ…
Harbin Institute of Technology（哈尔滨工业大学）；University of Oulu（奥卢大学）
The newly emerging federated learning (FL) framework offers a new way to train machine learning models in a privacy-preserving manner. However, traditional FL algorithms are based on an event-triggered aggregation, which suffers from stragglers and communication overhead issues. To address these issues, in this paper, we present a time-triggered FL algorithm (TT-Fed) over wireless networks, which is a generalized form of classic synchronous and asynchronous FL. Taking the constrained resource and unreliable nature of wireless communication into account, we jointly study the user selection and …
DP-EMAR: A Differentially Private Framework for Autonomous Model Weight Repair in Federated IoT Systems
Research Scholar, Department of CSE（计算机科学与工程系研究学者）；Professor, Department of CSE（计算机科学与工程系教授）；PES University, RR Nagar, Bangalore, India（印度班加罗尔RR Nagar的PES大学）
Quantum Cloud and Distributed Systems (qCLOUDS) Lab, School of Computing and Information Systems, The University of Melbourne, Australia（量子云与分布式系统实验室，计算机与信息系统学院，墨尔本大学，澳大利亚）；Department of Computer Science and Engineering, University of Visvesvaraya College of Engineering, Bangalore University, India（计算机科学与工程系，维萨瓦拉亚工程学院，班加罗尔大学，印度）
Synergizing Foundation Models and Federated Learning: A Survey
Uppsala University（乌普萨拉大学）；University College London（伦敦大学学院）；University of Liverpool（利物浦大学）；Eindhoven University of Technology（埃因霍温理工大学）；The University of Hong Kong（香港大学）；Research Institutes of Sweden（瑞典研究机构）
Intelligent Agents for Auction-based Federated Learning: A Survey
School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Department of Electrical and Computer Engineering, The University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）；Department of Computer Science, Bar-Ilan University（巴伊兰大学计算机科学系）
Towards Family-Grouped Hierarchical Federated Learning on Sub-5KB Models: A Feasibility Study of Privacy-Preserving ECG Monitoring for Ultra-Resource-Constrained Wearables
Topology-Aware Differential Privacy in Hierarchical Federated Learning
Quantum Cloud Computing and Distributed Systems (qCLOUDS) Lab, School of Computing and Information Systems, The University of Melbourne（量子云计算与分布式系统实验室，计算与信息系统学院，墨尔本大学）
Hierarchical Split Federated Learning: Convergence Analysis and System Optimization
Department of Electrical and Electronic Engineering, University of Hong Kong（香港大学电子与电气工程系）；Institute of Space Internet, Fudan University（复旦大学空间互联网研究所）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Faculty of Applied Sciences, Macao Polytechnic University（澳门理工学院应用科学系）；School of Computer Engineering, Nanyang Technological University（南洋理工大学计算机工程学院）
H-FLTN: A Privacy-Preserving Hierarchical Framework for Electric Vehicle Spatio-Temporal Charge Prediction
School of Computer Science, Queensland University of Technology（昆士兰科技大学计算机科学学院）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织数据61部）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
BASIRA Lab, Imperial-X (I-X) and Department of Computing（BASIRA实验室、Imperial-X（I-X）和计算系）；Imperial College London, London, United Kingdom（伦敦帝国学院）
Federated learning (FL) has emerged as a key paradigm for collaborative model training across multiple clients without sharing raw data, enabling privacy-preserving applications in areas such as radiology and pathology. However, works on collaborative training across clients with fundamentally different neural architectures and non-identically distributed datasets remain scarce. Existing FL frameworks face several limitations. Despite claiming to support architectural heterogeneity, most recent FL methods only tolerate variants within a single model family (e.g., shallower, deeper, or wider CN…
