本文针对联邦学习中基于 GAN 的特征推断攻击，提出了 Anti-GAN 防御框架，通过结合 GAN 生成的伪造图像与真实图像来操纵视觉特征，有效防止攻击者学习私有数据分布，同时保持模型准确性。
摘要：
联邦学习（FL）是一种去中心化的模型训练框架，旨在合并孤立的数据孤岛同时维护数据隐私。然而，最近的研究表明，基于生成对抗网络（GAN）的攻击可以在 FL 中被用来学习私有数据集的分布并重建可识别的图像。在本文中，我们探讨了针对 FL 中基于 GAN 攻击的防御方法，并提出了一个名为 Anti-GAN 的框架，以防止攻击者学习受害者数据的真实分布。Anti-GAN 的核心思想是操纵私有训练图像的视觉特征，使其即使被攻击者恢复，在人眼看来也无法区分。具体而言，Anti-GAN 将私有数据集投影到 GAN 的生成器上，并将生成的伪造图像与真实图像结合以创建训练数据集，随后将其用于联邦模型训练。实验结果表明，Anti-GAN 在防止攻击者学习私有图像分布方面是有效的，同时对联邦模型的准确性造成的损害极小。
英文摘要：
Federated learning (FL) is a decentralized model training framework that aims to merge isolated data islands while maintaining data privacy. However, recent studies have revealed that Generative Adversarial Network (GAN) based attacks can be employed in FL to learn the distribution of private datasets and reconstruct recognizable images. In this paper, we exploit defenses against GAN-based attacks in FL and propose a framework, Anti-GAN, to prevent attackers from learning the real distribution of the victim's data. The core idea of Anti-GAN is to manipulate the visual features of private training images to make them indistinguishable to human eyes even restored by attackers. Specifically, Anti-GAN projects the private dataset onto a GAN's generator and combines the generated fake images with the actual images to create the training dataset, which is then used for federated model training. The experimental results demonstrate that Anti-GAN is effective in preventing attackers from learning the distribution of private images while causing minimal harm to the accuracy of the federated model.
Federated learning (FL) enables the training of models among distributed clients without compromising the privacy of training datasets, while the invisibility of clients datasets and the training process poses a variety of security threats. This survey provides the taxonomy of poisoning attacks and experimental evaluation to discuss the need for robust FL.
Machine learning has been applied to almost all fields of computer science over the past decades. The introduction of GANs allowed for new possibilities in fields of medical research and text prediction. However, these new fields work with ever more privacy-sensitive data. In order to maintain user privacy, a combination of federated learning, differential privacy and GANs can be used to work with private data without giving away a users' privacy. Recently, two implementations of such combinations have been published: DP-Fed-Avg GAN and GS-WGAN. This paper compares their performance and introd…
FedMID: A Data-Free Method for Using Intermediate Outputs as a Defense Mechanism Against Poisoning Attacks in Federated Learning
School of Computing, KAIST（韩国科学技术院计算机学院）；Samsung Research（三星研究院）；Data Science Group, Institute for Basic Science（基础科学研究院数据科学组）；Max Planck Institute for Security and Privacy（马克斯·普朗克安全与隐私研究所）
Gradient inversion attack (or input recovery from gradient) is an emerging threat to the security and privacy preservation of Federated learning, whereby malicious eavesdroppers or participants in the protocol can recover (partially) the clients' private data. This paper evaluates existing attacks and defenses. We find that some attacks make strong assumptions about the setup. Relaxing such assumptions can substantially weaken these attacks. We then evaluate the benefits of three proposed defense mechanisms against gradient inversion attacks. We show the trade-offs of privacy leakage and data …
The goal of federated learning (FL) is to train one global model by aggregating model parameters updated independently on edge devices without accessing users' private data. However, FL is susceptible to backdoor attacks where a small fraction of malicious agents inject a targeted misclassification behavior in the global model by uploading polluted model updates to the server. In this work, we propose DifFense, an automated defense framework to protect an FL system from backdoor attacks by leveraging differential testing and two-step MAD outlier detection, without requiring any previous knowle…
Tongji University（同济大学）；Pennsylvania State University（宾夕法尼亚州立大学）
Federated Learning (FL) is a popular distributed machine learning paradigm that enables jointly training a global model without sharing clients' data. However, its repetitive server-client communication gives room for backdoor attacks with aim to mislead the global model into a targeted misprediction when a specific trigger pattern is presented. In response to such backdoor threats on federated learning, various defense measures have been proposed. In this paper, we study whether the current defense mechanisms truly neutralize the backdoor threats from federated learning in a practical setting…
Carnegie Mellon University（卡内基梅隆大学）；University of Washington（华盛顿大学）
Backdoor attacks are dangerous and difficult to prevent in federated learning (FL), where training data is sourced from untrusted clients over long periods of time. These difficulties arise because: (a) defenders in FL do not have access to raw training data, and (b) a new phenomenon we identify called backdoor leakage causes models trained continuously to eventually suffer from backdoors due to cumulative errors in defense mechanisms. We propose shadow learning, a framework for defending against backdoor attacks in the FL setting under long-range training. Shadow learning trains two models in…
Federated learning, as a distributed learning that conducts the training on the local devices without accessing to the training data, is vulnerable to Byzatine poisoning adversarial attacks. We argue that the federated learning model has to avoid those kind of adversarial attacks through filtering out the adversarial clients by means of the federated aggregation operator. We propose a dynamic federated aggregation operator that dynamically discards those adversarial clients and allows to prevent the corruption of the global learning model. We assess it as a defense against adversarial attacks …
Healthcare IoMT systems are becoming intelligent, miniaturized, and more integrated into daily life. As for the distributed devices in the IoMT, federated learning has become a topical area with cloud-based training procedures when meeting data security. However, the distribution of IoMT has the risk of protection from data poisoning attacks. Poisoned data can be fabricated by falsifying medical data, which urges a security defense to IoMT systems. Due to the lack of specific labels, the filtering of malicious data is a unique unsupervised scenario. One of the main challenges is finding robust…
Defending Against Sophisticated Poisoning Attacks with RL-based Aggregation in Federated Learning
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算高精尖创新中心）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）
FedSurrogate: Backdoor Defense in Federated Learning via Layer Criticality and Surrogate Replacement
The University of Manchester, UK（英国曼彻斯特大学）；Institute for Infocomm Research, A*STAR, Singapore（新加坡信息通信研究所）；Monash University, Australia（澳大利亚墨尔本大学）；COSIC, KU Leuven, Belgium（比利时列日大学COSIC实验室）
Federated learning is a recently proposed paradigm that enables multiple clients to collaboratively train a joint model. It allows clients to train models locally, and leverages the parameter server to generate a global model by aggregating the locally submitted gradient updates at each round. Although the incentive model for federated learning has not been fully developed, it is supposed that participants are able to get rewards or the privilege to use the final global model, as a compensation for taking efforts to train the model. Therefore, a client who does not have any local data has the …
Mississippi State University（密西西比州立大学）；University of South Florida（南佛罗里达大学）；University of Oklahoma（俄克拉荷马大学）
Federated learning (FL) provides a high efficient decentralized machine learning framework, where the training data remains distributed at remote clients in a network. Though FL enables a privacy-preserving mobile edge computing framework using IoT devices, recent studies have shown that this approach is susceptible to poisoning attacks from the side of remote clients. To address the poisoning attacks on FL, we provide a \textit{two-phase} defense algorithm called {Lo}cal {Ma}licious Facto{r} (LoMar). In phase I, LoMar scores model updates from each remote client by measuring the relative dist…
FedBAP: Backdoor Defense via Benign Adversarial Perturbation in Federated Learning
School of Cyber Science and Engineering, Wuhan University（武汉大学计算机科学与工程学院）；Department of Computer and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机与人工智能学院）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）
A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Zhejiang University（浙江大学）；AntGroup（蚂蚁集团）；Ulsan National Institute of Science and Technology（蔚山国立科学技术院）；The University of Hong Kong（香港大学）
The decentralized nature of federated learning makes detecting and defending against adversarial attacks a challenging task. This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task. Unlike existing works, we allow non-malicious clients to have correctly labeled samples from the targeted tasks. We conduct a comprehensive study of backdoor attacks and defenses for the EMNIST dataset, a real-life, user-partitioned, and non-iid dataset. W…
Sparsification Under Siege: Dual-Level Defense Against Poisoning in Communication-Efficient Federated Learning
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Cyber Science and Technology at Beihang University（北京航空航天大学网络安全科学与技术学院）；School of Cyberspace Science and Technology, Beijing Jiaotong University（北京交通大学网络空间科学与技术学院）；Beijing Key Laboratory of Security and Privacy in Intelligent Transportation（北京智能交通安全与隐私重点实验室）；Zhongguancun Laboratory（中关村实验室）；School of Computing and Information,University of Pittsburgh（匹兹堡大学计算与信息学院）
SoK: Benchmarking Poisoning Attacks and Defenses in Federated Learning
Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）；Jinan University（暨南大学）
Federated Learning (FL) is a distributed learning paradigm that enables different parties to train a model together for high quality and strong privacy protection. In this scenario, individual participants may get compromised and perform backdoor attacks by poisoning the data (or gradients). Existing work on robust aggregation and certified FL robustness does not study how hardening benign clients can affect the global model (and the malicious clients). In this work, we theoretically analyze the connection among cross-entropy loss, attack success rate, and clean accuracy in this setting. Moreo…
Robust Federated Learning with Confidence-Weighted Filtering and GAN-Based Completion under Noisy and Incomplete Data
Computer Engineering Department, İstanbul Commerce University（伊斯坦布尔商业大学计算机工程系）；Turkcell Teknoloji（Turkcell技术公司）；Grid Communications and Security Group, Electrification and Energy Infrastructures Division, Oak Ridge National Laboratory（橡树岭国家实验室电网安全与能源基础设施部门）
针对联邦学习中的噪声标签、类别缺失与不均衡问题，提出结合自适应噪声清洗、条件 GAN 合成补全和鲁棒训练的隐私保护框架，并在 MNIST 与 Fashion-MNIST 上验证其性能提升。
Real-world data is usually segmented by attributes and distributed across different parties. Federated learning empowers collaborative training without exposing local data or models. As we demonstrate through designed attacks, even with a small proportion of corrupted data, an adversary can accurately infer the input attributes. We introduce an adversarial learning based procedure which tunes a local model to release privacy-preserving intermediate representations. To alleviate the accuracy decline, we propose a defense method based on the forward-backward splitting algorithm, which respective…
FilterFL: Knowledge Filtering-based Data-Free Backdoor Defense for Federated Learning
East China Normal University, Shanghai, China（华东师范大学）；Singapore Management University, Singapore（新加坡管理学院）；Nanyang Technological University, Singapore（南洋理工大学）；National University of Singapore, Singapore（国立新加坡大学）
City University of Hong Kong（香港城市大学）；Monash University（莫纳什大学）；Tencent（腾讯）；Nanyang Technological University（南洋理工大学）
Semi-supervised Federated Learning (SSFL) has recently drawn much attention due to its practical consideration, i.e., the clients may only have unlabeled data. In practice, these SSFL systems implement semi-supervised training by assigning a "guessed" label to the unlabeled data near the labeled data to convert the unsupervised problem into a fully supervised problem. However, the inherent properties of such semi-supervised training techniques create a new attack surface. In this paper, we discover and reveal a simple yet powerful poisoning attack against SSFL. Our attack utilizes the natural …
VinUniversity（文大大学）；Hanoi University of Science and Technology（河内科技大学）
Federated learning (FL) is a machine learning (ML) approach that allows the use of distributed data without compromising personal privacy. However, the heterogeneous distribution of data among clients in FL can make it difficult for the orchestration server to validate the integrity of local model updates, making FL vulnerable to various threats, including backdoor attacks. Backdoor attacks involve the insertion of malicious functionality into a targeted model through poisoned updates from malicious clients. These attacks can cause the global model to misbehave on specific inputs while appeari…
Defending the Edge: Representative-Attention Defense against Backdoor Attacks in Federated Learning
Dept. of Computer Science Faculty of Engineering University of Manchester Manchester, United Kingdom；Dept. of Computer Science Faculty of Engineering University of Manchester \& MBZUAI Abu Dhabi, United Arab Emirates；Dept. of Computer Science Faculty of Engineering University of Manchester \& KU Leuven Manchester, United Kingdom
Poisoning Attacks and Defenses to Federated Unlearning
ShanghaiTech University（上海科技大学）；Xidian University（西安电子科技大学）；North Carolina State University（北卡罗来纳州立大学）；University of North Texas（北德克萨斯大学）；University of Louisville（路易斯维尔大学）
Federated learning (FL) is a popular distributed learning framework that can reduce privacy risks by not explicitly sharing private data. However, recent works demonstrated that sharing model updates makes FL vulnerable to inference attacks. In this work, we show our key observation that the data representation leakage from gradients is the essential cause of privacy leakage in FL. We also provide an analysis of this observation to explain how the data presentation is leaked. Based on this observation, we propose a defense against model inversion attack in FL. The key idea of our defense is le…
The Ohio State University（俄亥俄州立大学）；Duke University（杜克大学）
In federated learning, multiple client devices jointly learn a machine learning model: each client device maintains a local model for its local training dataset, while a master device maintains a global model via aggregating the local models from the client devices. The machine learning community recently proposed several federated learning methods that were claimed to be robust against Byzantine failures (e.g., system failures, adversarial manipulations) of certain client devices. In this work, we perform the first systematic study on local model poisoning attacks to federated learning. We as…
University of Maryland（马里兰大学）；IBM Research（IBM研究院）；University of Virginia（弗吉尼亚大学）
Due to its distributed methodology alongside its privacy-preserving features, Federated Learning (FL) is vulnerable to training time adversarial attacks. In this study, our focus is on backdoor attacks in which the adversary's goal is to cause targeted misclassifications for inputs embedded with an adversarial trigger while maintaining an acceptable performance on the main learning task at hand. Contemporary defenses against backdoor attacks in federated learning require direct access to each individual client's update which is not feasible in recent FL settings where Secure Aggregation is dep…
