Federated learning enables a cluster of decentralized mobile devices at the edge to collaboratively train a shared machine learning model, while keeping all the raw training samples on device. This decentralized training approach is demonstrated as a practical solution to mitigate the risk of privacy leakage. However, enabling efficient FL deployment at the edge is challenging because of non-IID training data distribution, wide system heterogeneity and stochastic-varying runtime effects in the field. This paper jointly optimizes time-to-convergence and energy efficiency of state-of-the-art FL use cases by taking into account the stochastic nature of edge execution. We propose AutoFL by tailor-designing a reinforcement learning algorithm that learns and determines which K participant devices and per-device execution targets for each FL model aggregation round in the presence of stochastic runtime variance, system and data heterogeneity. By considering the unique characteristics of FL edge deployment judiciously, AutoFL achieves 3.6 times faster model convergence time and 4.7 and 5.2 times higher energy efficiency for local clients and globally over the cluster of K participants, respectively.
Automated Federated Learning in Mobile Edge Networks -- Fast Adaptation and Convergence
Singapore University of Design and Technology（新加坡科技设计大学）；East China Normal University（华东师范大学）；University of Electronic Science and Technology of China（电子科技大学）；Beihang University（北京航空航天大学）
Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（教育部新一代人工智能技术及其交叉应用重点实验室（东南大学））
Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology（国家并行与分布式计算实验室，计算机科学与技术学院，国防科技大学）
Tackling Privacy Heterogeneity in Differentially Private Federated Learning
Department of Information Engineering, The Chinese University of Hong Kong（信息工程系，香港中文大学）；School of Science and Engineering, Shenzhen Institute of Artificial Intelligence and Robotics for Society（科学与工程学院，深圳人工智能与机器人社会研究院）
A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning
McGill University（麦吉尔大学）；Mila-Quebec AI Institute（魁北克AI研究院）；The University of Adelaide（阿德莱德大学）；University of Southern Queensland（南方昆士兰大学）；Deakin University（德肯大学）；Monash University（墨尔本大学）
Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Hong Kong Baptist University（香港浸会大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Combating Exacerbated Heterogeneity for Robust Models in Federated Learning
Hong Kong Baptist University（香港浸会大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Sydney（悉尼大学）；Tsinghua University（清华大学）
Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院）；School of Systems Science and Engineering, Sun Yat-sen University（中山大学系统科学与工程学院）
Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
Key Lab of High Confidence Software Technologies (Peking University)（高可信软件技术重点实验室（北京大学））；State Key Laboratory of Networking and Switching Technology (BUPT)（网络与交换技术国家重点实验室（北京邮电大学））；Department of Computer Science and Technology, Peking University（北京大学计算机科学技术系）；Microsoft Research（微软研究院）
FedSPM: Routing-Enabled Federated Learning under Dual Heterogeneity via Semiparametric Mixture
Institute of Statistics and Big Data, Renmin University of China（中国人民大学统计与大数据研究院）；Department of Statistics and Actuarial Science, University of Waterloo（滑铁卢大学统计与精算科学系）
HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems
Department of Electrical and Electronic Engineering, University of Hong Kong（香港大学电子与电气工程系）；Institute of Space Internet, Fudan University（复旦大学空间互联网研究所）；School of Computer Science, Fudan University（复旦大学计算机科学学院）
Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity
Shanghai Jiao Tong University（上海交通大学）；Nanjing University of Science and Technology（南京理工大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织数据61研究所）；Princeton University（普林斯顿大学）
FedFed: Feature Distillation against Data Heterogeneity in Federated Learning
Beihang University（北京航空航天大学）；Chinese University of Hong Kong（香港中文大学）；University of Science and Technology of China（中国科学技术大学）；The University of Sydney（悉尼大学）；Hong Kong Baptist University（香港浸会大学）；Kunming University of Science and Technology（昆明理工大学）
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning
Meta AI
联邦学习的一个经常提到的挑战是异质性的存在。\emph{Data heterogeneity}是指来自不同客户端的数据可能遵循非常不同的分布。\emph{System heterogeneity}是指具有不同系统能力的客户端设备。大量的联邦优化方法可以解决这一挑战。在文献中，经验评估通常从随机初始化开始联邦训练。然而，在联邦学习的许多实际应用中，服务器可以访问训练任务的代理数据，这些数据可以用于在开始联合训练之前预训练模型。使用四个标准的联邦学习基准数据集，我们实证研究了从联邦学习中的预训练模型开始的影响。不出所料，从预先训练的模型开始减少了达到目标出错率所需的训练时间，并且能够训练比从随机初始化开始时更准确的模型（高达40%）。令人惊讶的是，我们还发现，从预先训练的初始化开始联邦学习可以减少数据和系统异构性的影响。我们建议未来的工作提出和评估联邦优化方法，以评估从随机和预先训练的初始化开始时的性能。这项研究提出了几个问题，进一步了解联邦优化中的异构性的作用。\footnote{Our code is available at: \url{https://github.com/facebookresearch/where_to_begin}}
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）
Tackling Heterogeneity in Federated Learning via Variance-Reduced Boltzmann Sampling within Homogeneous Social Coalitions
Department of Mathematical Sciences, Politecnico di Torino（数学科学系，都灵理工学院）；Istituto Nazionale di Fisica Nucleare, Sezione di Torino（国家核物理研究所，都灵分所）；Department of Electronics and Telecommunications, Politecnico di Torino（电子与电信系，都灵理工学院）
