In the resource management of wireless networks, Federated Learning has been used to predict handovers. However, non-independent and identically distributed data degrade the accuracy performance of such predictions. To overcome the problem, Federated Learning can leverage data clustering algorithms and build a machine learning model for each cluster. However, traditional data clustering algorithms, when applied to the handover prediction, exhibit three main limitations: the risk of data privacy breach, the fixed shape of clusters, and the non-adaptive number of clusters. To overcome these limitations, in this paper, we propose a three-phased data clustering algorithm, namely: generative adversarial network-based clustering, cluster calibration, and cluster division. We show that the generative adversarial network-based clustering preserves privacy. The cluster calibration deals with dynamic environments by modifying clusters. Moreover, the divisive clustering explores the different number of clusters by repeatedly selecting and dividing a cluster into multiple clusters. A baseline algorithm and our algorithm are tested on a time series forecasting task. We show that our algorithm improves the performance of forecasting models, including cellular network handover, by 43%.
Federated Learning (FL) is a setting where multiple parties with distributed data collaborate in training a joint Machine Learning (ML) model while keeping all data local at the parties. Federated clustering is an area of research within FL that is concerned with grouping together data that is globally similar while keeping all data local. We describe how this area of research can be of interest in itself, or how it helps addressing issues like non-independently-identically-distributed (i.i.d.) data in supervised FL frameworks. The focus of this work, however, is an extension of the federated …
Carnegie Mellon University（卡内基梅隆大学）；University of Novi Sad（诺维萨德大学）
We propose a parametric family of algorithms for personalized federated learning with locally convex user costs. The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized via a sum-of-norms penalty, weighted by a penalty parameter $λ$. The proposed approach enables "automatic" model clustering, without prior knowledge of the hidden cluster structure, nor the number of clusters. Analytical bounds on the weight parameter, that lead to simultaneous personalization, generalization and automatic model clustering a…
In this work, we explore the unique challenges -- and opportunities -- of unsupervised federated learning (FL). We develop and analyze a one-shot federated clustering scheme, $k$-FED, based on the widely-used Lloyd's method for $k$-means clustering. In contrast to many supervised problems, we show that the issue of statistical heterogeneity in federated networks can in fact benefit our analysis. We analyse $k$-FED under a center separation assumption and compare it to the best known requirements of its centralized counterpart. Our analysis shows that in heterogeneous regimes where the number o…
A new type of federated clustering: A non-model-sharing approach
Center for Artificial Intelligence Research（人工智能研究中心）；Tsukuba Institute for Advanced Research（茨口高级研究机构）；University of Tsukuba（茨口大学）；Graduate School of Science and Technology（科学与技术研究生院）；Institute of Systems and Information Engineering（系统与信息工程研究所）
National Institute of Technology Kurukshetra（国立库鲁克舍特拉理工学院）
In recent times, federated machine learning has been very useful in building intelligent intrusion detection systems for IoT devices. As IoT devices are equipped with a security architecture vulnerable to various attacks, these security loopholes may bring a risk during federated training of decentralized IoT devices. Adversaries can take control over these IoT devices and inject false gradients to degrade the global model performance. In this paper, we have proposed an approach that detects the adversaries with the help of a clustering algorithm. After clustering, it further rewards the clien…
Reinforcement Federated Learning Method Based on Adaptive OPTICS Clustering
Beijing Key Laboratory of Intelligent Communication Software and Multimedia（北京智能通信软件与多媒体重点实验室）；School of Computer Science (National Pilot Software Engineering School)（计算机学院（国家级试点软件学院））；Beijing University of Posts and Telecommunications（北京邮电大学）
University of Alberta（阿尔伯塔大学）；University of Hradec Králové（赫拉德茨-克拉洛韦大学）
Electrical load prediction has become an integral part of power system operation. Deep learning models have found popularity for this purpose. However, to achieve a desired prediction accuracy, they require huge amounts of data for training. Sharing electricity consumption data of individual households for load prediction may compromise user privacy and can be expensive in terms of communication resources. Therefore, edge computing methods, such as federated learning, are gaining more importance for this purpose. These methods can take advantage of the data without centrally storing it. This p…
Clustering in graphs has been a well-known research problem, particularly because most Internet and social network data is in the form of graphs. Organizations widely use spectral clustering algorithms to find clustering in graph datasets. However, applying spectral clustering to a large dataset is challenging due to computational overhead. While the distributed spectral clustering algorithm exists, they face the problem of data privacy and increased communication costs between the clients. Thus, in this paper, we propose a spectral clustering algorithm using federated learning (FL) to overcom…
Institut LIST, CEA（法国原子能与替代能源委员会 LIST 研究所）；Université Paris-Saclay（巴黎萨克雷大学）
Federated learning enables different parties to collaboratively build a global model under the orchestration of a server while keeping the training data on clients' devices. However, performance is affected when clients have heterogeneous data. To cope with this problem, we assume that despite data heterogeneity, there are groups of clients who have similar data distributions that can be clustered. In previous approaches, in order to cluster clients the server requires clients to send their parameters simultaneously. However, this can be problematic in a context where there is a significant nu…
Federated learning is a model for privacy without revealing private data by transfer models instead of personal and private data from local client devices. While, in the global model, it's crucial to recognize each local data is normal. This paper suggests one method to separate normal locals and abnormal locals by Euclidean similarity clustering of vectors extracted by inputting dummy data in local models. In a federated classification model, this method divided locals into normal and abnormal.
Federated Learning (FL) under distributed concept drift is a largely unexplored area. Although concept drift is itself a well-studied phenomenon, it poses particular challenges for FL, because drifts arise staggered in time and space (across clients). To the best of our knowledge, this work is the first to explicitly study data heterogeneity in both dimensions. We first demonstrate that prior solutions to drift adaptation that use a single global model are ill-suited to staggered drifts, necessitating multiple-model solutions. We identify the problem of drift adaptation as a time-varying clust…
FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；The Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Guangdong Provincial Key Laboratory of Future Networks of Intelligence（广东省未来智能网络重点实验室）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）；School of Engineering, Westlake University（西湖大学工程学院）
Privacy-preserving Continual Federated Clustering via Adaptive Resonance Theory
Osaka Metropolitan University（大阪公立大学）；Okayama University（冈山大学）；Universiti Malaya（马来亚大学）；Southern University of Science and Technology（南方科技大学）；Tokyo Metropolitan University（东京都立大学）
University of Michigan（密歇根大学）；University of Cambridge（剑桥大学）；Nokia Bell Labs（诺基亚贝尔实验室）
Federated learning is generally used in tasks where labels are readily available (e.g., next word prediction). Relaxing this constraint requires design of unsupervised learning techniques that can support desirable properties for federated training: robustness to statistical/systems heterogeneity, scalability with number of participants, and communication efficiency. Prior work on this topic has focused on directly extending centralized self-supervised learning techniques, which are not designed to have the properties listed above. To address this situation, we propose Orchestra, a novel unsup…
College of Computing and Informatics, Sungkyunkwan University（成均馆大学计算与信息学院）；Bio-Medical IT Convergence Research Division, Electronics and Telecommunications Research Institute（电子通信研究院生物医学IT融合研究部）；Samsung Medical Center（三星医疗中心）
While machine learning techniques are being applied to various fields for their exceptional ability to find complex relations in large datasets, the strengthening of regulations on data ownership and privacy is causing increasing difficulty in its application to medical data. In light of this, Federated Learning has recently been proposed as a solution to train on private data without breach of confidentiality. This conservation of privacy is particularly appealing in the field of healthcare, where patient data is highly confidential. However, many studies have shown that its assumption of Ind…
School of Automation Science and Engineering, South China University of Technology（华南理工大学自动化科学与工程学院）；School of Software, Dalian University of Technology（大连理工大学软件学院）；School of Telecommunications Engineering, Xidian University（西安电子科技大学电信工程学院）
A Joint Gradient and Loss Based Clustered Federated Learning Design
University of Miami（迈阿密大学）；Frost Institute for Data Science and Computing, University of Miami（迈阿密大学弗罗斯特数据科学与计算研究所）；Zhejiang University（浙江大学）；North Carolina State University（北卡罗来纳州立大学）
Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；School of Engineering, Westlake University（西湖大学工学院）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）
One-Shot Federated Clustering of Non-Independent Completely Distributed Data
School of Computer Science and Technology, Guangdong University of Technology（广东工业大学计算机科学与技术学院）；Department of Computer Science, Beijing Normal-Hong Kong Baptist University（北京师范大学-香港 Baptist大学计算机科学系）；Department of Computer Science, Harbin Institute of Technology（哈尔滨工业大学计算机科学系）
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；University of Southern California（南加州大学）
We consider a foundational unsupervised learning task of $k$-means data clustering, in a federated learning (FL) setting consisting of a central server and many distributed clients. We develop SecFC, which is a secure federated clustering algorithm that simultaneously achieves 1) universal performance: no performance loss compared with clustering over centralized data, regardless of data distribution across clients; 2) data privacy: each client's private data and the cluster centers are not leaked to other clients and the server. In SecFC, the clients perform Lagrange encoding on their local d…
Vertical federated learning (VFL), where data features are stored in multiple parties distributively, is an important area in machine learning. However, the communication complexity for VFL is typically very high. In this paper, we propose a unified framework by constructing coresets in a distributed fashion for communication-efficient VFL. We study two important learning tasks in the VFL setting: regularized linear regression and $k$-means clustering, and apply our coreset framework to both problems. We theoretically show that using coresets can drastically alleviate the communication complex…
Institute of Communication Acoustics, Ruhr-Universit\"at Bochum 44780 Bochum, Germany
In this paper we introduce a realistic and challenging, multi-source and multi-room acoustic environment and an improved algorithm for the estimation of source-dominated microphone clusters in acoustic sensor networks. Our proposed clustering method is based on a single microphone per node and on unsupervised clustered federated learning which employs a light-weight autoencoder model. We present an improved clustering control strategy that takes into account the variability of the acoustic scene and allows the estimation of a dynamic range of clusters using reduced amounts of training data. Th…
Federated learning (FL) is a well established method for performing machine learning tasks over massively distributed data. However in settings where data is distributed in a non-iid (not independent and identically distributed) fashion -- as is typical in real world situations -- the joint model produced by FL suffers in terms of test set accuracy and/or communication costs compared to training on iid data. We show that learning a single joint model is often not optimal in the presence of certain types of non-iid data. In this work we present a modification to FL by introducing a hierarchical…
Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；LSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Noah’s Ark Lab, Huawei（华为诺亚实验室）
A Semi-Supervised Federated Learning Framework with Hierarchical Clustering Aggregation for Heterogeneous Satellite Networks
School of Computer Science and Artificial Intelligence, Wuhan University of Technology（计算机科学与人工智能学院，武汉科技大学）；Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology（交通物联网湖北省重点实验室，武汉科技大学）；School of Engineering, Swinburne University of Technology（工程学院，斯威本科技大学）；School of Computer Science and Technology, Tongji University（计算机科学与技术学院，同济大学）；State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）
Dual-Segment Clustering Strategy for Hierarchical Federated Learning in Heterogeneous Wireless Environments
College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Tongji Hospital, Tongji University（同济大学附属同济医院）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61研究院）；Guangzhou Huatu Information Technology Co., Ltd.（广州华图信息技术有限公司）；School of Electrical and Computer Engineering, The University of Sydney（悉尼大学电气与计算机工程学院）
