Wireless ad hoc federated learning (WAFL) is a fully decentralized collaborative machine learning framework organized by opportunistically encountered mobile nodes. Compared to conventional federated learning, WAFL performs model training by weakly synchronizing the model parameters with others, and this shows great resilience to a poisoned model injected by an attacker. In this paper, we provide our theoretical analysis of the WAFL's resilience against model poisoning attacks, by formulating the force balance between the poisoned model and the legitimate model. According to our experiments, we confirmed that the nodes directly encountered the attacker has been somehow compromised to the poisoned model but other nodes have shown great resilience. More importantly, after the attacker has left the network, all the nodes have finally found stronger model parameters combined with the poisoned model. Most of the attack-experienced cases achieved higher accuracy than the no-attack-experienced cases.
Resilience in Online Federated Learning: Mitigating Model-Poisoning Attacks via Partial Sharing
Norwegian University of Science and Technology（挪威科技大学）；Aalto University（阿尔托大学）；Commonwealth Scientific and Industrial Research Organisation（联邦科学与工业研究组织）；University of Southern Denmark（南丹麦大学）
To study the resilience of distributed learning, the "Byzantine" literature considers a strong threat model where workers can report arbitrary gradients to the parameter server. Whereas this model helped obtain several fundamental results, it has sometimes been considered unrealistic, when the workers are mostly trustworthy machines. In this paper, we show a surprising equivalence between this model and data poisoning, a threat considered much more realistic. More specifically, we prove that every gradient attack can be reduced to data poisoning, in any personalized federated learning system w…
University of California Merced（加州大学默塞德分校）；Tagliatela College of Engineering（塔利亚泰拉工程学院）；University of New Haven（纽黑文大学）
State-of-the-art machine learning models are vulnerable to data poisoning attacks whose purpose is to undermine the integrity of the model. However, the current literature on data poisoning attacks is mainly focused on ad hoc techniques that are only applicable to specific machine learning models. Additionally, the existing data poisoning attacks in the literature are limited to either binary classifiers or to gradient-based algorithms. To address these limitations, this paper first proposes a novel model-free label-flipping attack based on the multi-modality of the data, in which the adversar…
Data Poisoning and Leakage Analysis in Federated Learning
Georgia Institute of Technology（佐治亚理工学院）；Fordham University（福特汉姆大学）；National Institute of Standards and Technology（美国国家标准与技术研究院）；Georgia Tech Research Institute（佐治亚理工研究院）
East China Normal University（华东师范大学）；The University of New South Wales（新南威尔士大学）；CSIRO-Data61（澳大利亚联邦科学与工业研究组织Data61）；The University of Adelaide（阿德莱德大学）；Northeastern University（东北大学）
With the rise of third parties in the machine learning pipeline, the service provider in "Machine Learning as a Service" (MLaaS), or external data contributors in online learning, or the retraining of existing models, the need to ensure the security of the resulting machine learning models has become an increasingly important topic. The security community has demonstrated that without transparency of the data and the resulting model, there exist many potential security risks, with new risks constantly being discovered. In this paper, we focus on one of these security risks -- poisoning attacks…
Federated learning (FL) is a machine learning technique that aims at training an algorithm across decentralized entities holding their local data private. Wireless mobile networks allow users to communicate with other fixed or mobile users. The road traffic network represents an infrastructure-based configuration of a wireless mobile network where the Connected and Automated Vehicles (CAV) represent the communicating entities. Applying FL in a wireless mobile network setting gives rise to a new threat in the mobile environment that is very different from the traditional fixed networks. The thr…
Zhejiang University of Technology（浙江工业大学）；Zhejiang University（浙江大学）
Deep neural networks are susceptible to poisoning attacks by purposely polluted training data with specific triggers. As existing episodes mainly focused on attack success rate with patch-based samples, defense algorithms can easily detect these poisoning samples. We propose DeepPoison, a novel adversarial network of one generator and two discriminators, to address this problem. Specifically, the generator automatically extracts the target class' hidden features and embeds them into benign training samples. One discriminator controls the ratio of the poisoning perturbation. The other discrimin…
Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience
Ecole Polytechnique Federale de Lausanne（洛桑联邦理工学院）；Hanoi University of Science and Technology（河内科技大学）；Griffith University（格里菲斯大学）；Humboldt-Universitat zu Berlin（柏林洪堡大学）
Khalifa University（哈利法大学）；Università degli Studi di Milano（米兰大学）
The major aim of this paper is to explain the data poisoning attacks using label-flipping during the training stage of the electroencephalogram (EEG) signal-based human emotion evaluation systems deploying Machine Learning models from the attackers' perspective. Human emotion evaluation using EEG signals has consistently attracted a lot of research attention. The identification of human emotional states based on EEG signals is effective to detect potential internal threats caused by insider individuals. Nevertheless, EEG signal-based human emotion evaluation systems have shown several vulnerab…
Privacy regulation laws, such as GDPR, impose transparency and security as design pillars for data processing algorithms. In this context, federated learning is one of the most influential frameworks for privacy-preserving distributed machine learning, achieving astounding results in many natural language processing and computer vision tasks. Several federated learning frameworks employ differential privacy to prevent private data leakage to unauthorized parties and malicious attackers. Many studies, however, highlight the vulnerabilities of standard federated learning to poisoning and inferen…
University of Science and Technology of China（中国科学技术大学）
Adversarial machine learning has attracted a great amount of attention in recent years. In a poisoning attack, the adversary can inject a small number of specially crafted samples into the training data which make the decision boundary severely deviate and cause unexpected misclassification. Due to the great importance and popular use of support vector machines (SVM), we consider defending SVM against poisoning attacks in this paper. We study two commonly used strategies for defending: designing robust SVM algorithms and data sanitization. Though several robust SVM algorithms have been propose…
Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, by design, MPC protocols faithfully compute the training functionality, which the adversarial ML community has shown to leak private information and can be tampered with in poisoning attacks. In this work, we argue that model ensembles, implemented in our framework called SafeNet, are a highly MPC-amenable way to avoid many adversarial ML attacks. The natural partitioning of data amongst owners in MPC training al…
FedFiTS: Fitness-Selected, Slotted Client Scheduling for Trustworthy Federated Learning in Healthcare AI
Department of Computer Science University of the Western Cape（计算机科学系，西开普敦大学）；Department of Computer Science Missouri University of Science and Technology（计算机科学系，密苏里科学与技术大学）
Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）
Federated recommendation systems can provide good performance without collecting users' private data, making them attractive. However, they are susceptible to low-cost poisoning attacks that can degrade their performance. In this paper, we develop a novel federated recommendation technique that is robust against the poisoning attack where Byzantine clients prevail. We argue that the key to Byzantine detection is monitoring of gradients of the model parameters of clients. We then propose a robust learning strategy where instead of using model parameters, the central server computes and utilizes…
Fed-Credit: Robust Federated Learning with Credibility Management
Beijing Normal University at Zhuhai（北京师范大学珠海校区）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）
SUSTech（南方科技大学）；The Hong Kong Polytechnic University（香港理工大学）；Hong Kong University of Science and Technology（香港科技大学）；University of Science and Technology of China（中国科学技术大学）
