Federated Learning (FL) presents a decentralized approach to model training in the agri-food sector and offers the potential for improved machine learning performance, while ensuring the safety and privacy of individual farms or data silos. However, the conventional FL approach has two major limitations. First, the heterogeneous data on individual silos can cause the global model to perform well for some clients but not all, as the update direction on some clients may hinder others after they are aggregated. Second, it is lacking with respect to the efficiency perspective concerning communication costs during FL and large model sizes. This paper proposes a new technical solution that utilizes network pruning on client models and aggregates the pruned models. This method enables local models to be tailored to their respective data distribution and mitigate the data heterogeneity present in agri-food data. Moreover, it allows for more compact models that consume less data during transmission. We experiment with a soybean yield forecasting dataset and find that this approach can improve inference performance by 15.5% to 20% compared to FedAvg, while reducing local model sizes by up to 84% and the data volume communicated between the clients and the server by 57.1% to 64.7%.
George Washington University（乔治华盛顿大学）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）
One of the biggest challenges in Federated Learning (FL) is that client devices often have drastically different computation and communication resources for local updates. To this end, recent research efforts have focused on training heterogeneous local models obtained by pruning a shared global model. Despite empirical success, theoretical guarantees on convergence remain an open question. In this paper, we present a unifying framework for heterogeneous FL algorithms with {\em arbitrary} adaptive online model pruning and provide a general convergence analysis. In particular, we prove that und…
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；Stevens Neuroimaging and Informatics Institute, University of Southern California（南加州大学史蒂文斯神经影像与信息学研究所）
Federated training of large deep neural networks can often be restrictive due to the increasing costs of communicating the updates with increasing model sizes. Various model pruning techniques have been designed in centralized settings to reduce inference times. Combining centralized pruning techniques with federated training seems intuitive for reducing communication costs -- by pruning the model parameters right before the communication step. Moreover, such a progressive model pruning approach during training can also reduce training times/costs. To this end, we propose FedSparsify, which pe…
Artificial neural network has achieved unprecedented success in a wide variety of domains such as classifying, predicting and recognizing objects. This success depends on the availability of big data since the training process requires massive and representative data sets. However, data collection is often prevented by privacy concerns and people want to take control over their sensitive information during both training and using processes. To address this problem, we propose a privacy-preserving method for the distributed system, Stochastic Channel-Based Federated Learning (SCBF), which enabl…
Iowa State University（爱荷华州立大学）；University of Minnesota（明尼苏达大学）
Federated Learning (FL) has emerged as a new paradigm for training machine learning models distributively without sacrificing data security and privacy. Learning models on edge devices such as mobile phones is one of the most common use cases for FL. However, Non-identical independent distributed~(non-IID) data in edge devices easily leads to training failures. Especially, over-parameterized machine learning models can easily be over-fitted on such data, hence, resulting in inefficient federated learning and poor model performance. To overcome the over-fitting issue, we proposed an adaptive dy…
Yale University（耶鲁大学）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Stanford Institute for Human-Centered Artificial Intelligence（斯坦福人类中心人工智能研究所）；Imperial College London（帝国理工学院）
Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers in a datacenter. To overcome this challenge, we propose PruneFL -- a novel FL approach with adaptive and distributed parameter pruning, which adapts the model size during FL to reduce both communication and computation overhead and minimize the overall training time, while maint…
Federated Learning (FL) is a privacy-preserving distributed deep learning paradigm that involves substantial communication and computation effort, which is a problem for resource-constrained mobile and IoT devices. Model pruning/sparsification develops sparse models that could solve this problem, but existing sparsification solutions cannot satisfy at the same time the requirements for low bidirectional communication overhead between the server and the clients, low computation overhead at the clients, and good model accuracy, under the FL assumption that the server does not have access to raw …
Automatic Speech Recognition models require large amount of speech data for training, and the collection of such data often leads to privacy concerns. Federated learning has been widely used and is considered to be an effective decentralized technique by collaboratively learning a shared prediction model while keeping the data local on different clients devices. However, the limited computation and communication resources on clients devices present practical difficulties for large models. To overcome such challenges, we propose Federated Pruning to train a reduced model under the federated set…
Hong Kong University of Science and Technology（香港科技大学）；University of Science and Technology China（中国科学技术大学）；University of Cambridge（剑桥大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）
Federated learning alleviates the privacy risk in distributed learning by transmitting only the local model updates to the central server. However, it faces challenges including statistical heterogeneity of clients' datasets and resource constraints of client devices, which severely impact the training performance and user experience. Prior works have tackled these challenges by combining personalization with model compression schemes including quantization and pruning. However, the pruning is data-dependent and thus must be done on the client side which requires considerable computation cost.…
University of Alberta（阿尔伯塔大学）；University of Hradec Králové（赫拉德茨-克拉洛韦大学）
Electrical load prediction has become an integral part of power system operation. Deep learning models have found popularity for this purpose. However, to achieve a desired prediction accuracy, they require huge amounts of data for training. Sharing electricity consumption data of individual households for load prediction may compromise user privacy and can be expensive in terms of communication resources. Therefore, edge computing methods, such as federated learning, are gaining more importance for this purpose. These methods can take advantage of the data without centrally storing it. This p…
Electrical Load Forecasting over Multihop Smart Metering Networks with Federated Learning
Department of Electrical and Computer Engineering, University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校电气与计算机工程系）；Computer Science and Mathematics Division, Oak Ridge National Laboratory（橡树岭国家实验室计算机科学与数学 division）；Department of Electrical and Computer Engineering, Kansas State University（堪萨斯州立大学电气与计算机工程系）
Tsinghua University（清华大学）；Huawei Wireless Technology Lab（华为无线技术实验室）；Hong Kong University of Science and Technology（香港科技大学）
Federated learning (FL) has prevailed as an efficient and privacy-preserved scheme for distributed learning. In this work, we mainly focus on the optimization of computation and communication in FL from a view of pruning. By adopting layer-wise pruning in local training and federated updating, we formulate an explicit FL pruning framework, FedLP (Federated Layer-wise Pruning), which is model-agnostic and universal for different types of deep learning models. Two specific schemes of FedLP are designed for scenarios with homogeneous local models and heterogeneous ones. Both theoretical and exper…
Automatic Time Series Forecasting (TSF) model design which aims to help users to efficiently design suitable forecasting model for the given time series data scenarios, is a novel research topic to be urgently solved. In this paper, we propose AutoTS algorithm trying to utilize the existing design skills and design efficient search methods to effectively solve this problem. In AutoTS, we extract effective design experience from the existing TSF works. We allow the effective combination of design experience from different sources, so as to create an effective search space containing a variety o…
Federated Topic Model and Model Pruning Based on Variational Autoencoder
Beijing Key Laboratory of Intelligent Communication Software and Multimedia（智能通信软件与多媒体北京重点实验室）；Beijing University of Posts and Telecommunications（北京邮电大学）；School of Economics and Management（经济管理学院）
Exploring Federated Pruning for Large Language Models
The University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）；Sun Yat-sen University（中山大学）；Guangming Laboratory（光明实验室）；Hangzhou Dianzi University（杭州电子科技大学）
Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National University of Singapore（新加坡国立大学）；Salesforce AI Research（Salesforce AI研究部）；Squirrel AI（松鼠AI）
On Electric Vehicle Energy Demand Forecasting and the Effect of Federated Learning
Department of Informatics, University of Piraeus（信息学院，比雷埃克斯大学）；Center for Power and Energy Systems, INESC TEC（能源系统中心，INESC TEC）；Department of Statistics and Insurance Science, University of Piraeus（统计与保险科学系，比雷埃克斯大学）
The traditional approach in FL tries to learn a single global model collaboratively with the help of many clients under the orchestration of a central server. However, learning a single global model might not work well for all clients participating in the FL under data heterogeneity. Therefore, the personalization of the global model becomes crucial in handling the challenges that arise with statistical heterogeneity and the non-IID distribution of data. Unlike prior works, in this work we propose a new approach for obtaining a personalized model from a client-level objective. This further mot…
Federated Dynamic Modeling and Learning for Spatiotemporal Data Forecasting
COSYS-GRETTIA, Gustave Eiffel University, 77420 France（COSYS-GRETTIA，巴黎-伊夫林大学）；ENTPE, University of Lyon（ENTPE，里昂大学）；the LICIT-ECO7 University Gustave Eiffel, France（LICIT-ECO7 巴黎-伊夫林大学）
Every Client Is an Environment: Federated De-confounding for Spatio-Temporal Forecasting
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shanghai Jiao Tong University（上海交通大学）；Harbin Engineering University（哈尔滨工程大学）
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Guangzhou HKUST Fok Ying Tung Research Institute（广州香港科技大学霍英东研究院）；Baidu Research, Baidu Inc.（百度研究院，百度公司）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；BOSS Zhipin（BOSS直聘）
Northeast Electric Power University（东北电力大学）；State Grid Jilin Electric Power Co., Ltd. Jilin Power Supply Company（国网吉林省电力有限公司吉林供电公司）；Huazhong University of Science and Technology（华中科技大学）；Naval University of Engineering（海军工程大学）；Cranfield University（克兰菲尔德大学）
In a modern power system with an increasing proportion of renewable energy, wind power prediction is crucial to the arrangement of power grid dispatching plans due to the volatility of wind power. However, traditional centralized forecasting methods raise concerns regarding data privacy-preserving and data islands problem. To handle the data privacy and openness, we propose a forecasting scheme that combines federated learning and deep reinforcement learning (DRL) for ultra-short-term wind power forecasting, called federated deep reinforcement learning (FedDRL). Firstly, this paper uses the de…
