Federated learning (FL) enables collaborative model training without sharing raw data but is vulnerable to gradient inversion attacks (GIAs), where adversaries reconstruct private data from shared gradients. Existing defenses either incur impractical computational overhead for embedded platforms or fail to achieve privacy protection and good model utility at the same time. Moreover, many defenses can be easily bypassed by adaptive adversaries who have obtained the defense details. To address these limitations, we propose SVDefense, a novel defense framework against GIAs that leverages the truncated Singular Value Decomposition (SVD) to obfuscate gradient updates. SVDefense introduces three key innovations, a Self-Adaptive Energy Threshold that adapts to client vulnerability, a Channel-Wise Weighted Approximation that selectively preserves essential gradient information for effective model training while enhancing privacy protection, and a Layer-Wise Weighted Aggregation for effective model aggregation under class imbalance. Our extensive evaluation shows that SVDefense outperforms existing defenses across multiple applications, including image classification, human activity recognition, and keyword spotting, by offering robust privacy protection with minimal impact on model accuracy. Furthermore, SVDefense is practical for deployment on various resource-constrained embedded platforms. We will make our code publicly available upon paper acceptance.
Gradient inversion attack (or input recovery from gradient) is an emerging threat to the security and privacy preservation of Federated learning, whereby malicious eavesdroppers or participants in the protocol can recover (partially) the clients' private data. This paper evaluates existing attacks and defenses. We find that some attacks make strong assumptions about the setup. Relaxing such assumptions can substantially weaken these attacks. We then evaluate the benefits of three proposed defense mechanisms against gradient inversion attacks. We show the trade-offs of privacy leakage and data …
Shadow defense against gradient inversion attack in federated learning
Bioengineering Department, Imperial College London, London W12 0BZ, UK；School of Computer Engineering；Science, Shanghai university, Shanghai 200444, China；Bioengineering Department；Imperial-X, Imperial College London, London W12 7SL, UK；Lung Institute, Imperial College London, London SW7 2AZ, UK；Cardiovascular Research Centre, Royal Brompton Hospital, London SW3 6NP, UK；School of Biomedical Engineering \& Imaging Sciences, King's College London, London WC2R 2LS, UK
University of Sindh（信德大学）；Munster Technological University（芒斯特理工大学）；Nearby Computing（Nearby Computing公司）
The applications concerning vehicular networks benefit from the vision of beyond 5G and 6G technologies such as ultra-dense network topologies, low latency, and high data rates. Vehicular networks have always faced data privacy preservation concerns, which lead to the advent of distributed learning techniques such as federated learning. Although federated learning has solved data privacy preservation issues to some extent, the technique is quite vulnerable to model inversion and model poisoning attacks. We assume that the design of defense mechanism and attacks are two sides of the same coin. …
TIP: Resisting Gradient Inversion via Targeted Interpretable Perturbation in Federated Learning
Shanxi Key Laboratory of Industrial Internet Security, College of Computer Science and Technology（工业互联网安全山西省重点实验室，计算机科学与技术学院）；Taiyuan University of Technology（太原科技大学）
Gradient inversion attacks on federated learning systems reconstruct client training data from exchanged gradient information. To defend against such attacks, a variety of defense mechanisms were proposed. However, they usually lead to an unacceptable trade-off between privacy and model utility. Recent observations suggest that dropout could mitigate gradient leakage and improve model utility if added to neural networks. Unfortunately, this phenomenon has not been systematically researched yet. In this work, we thoroughly analyze the effect of dropout on iterative gradient inversion attacks. W…
Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks
Huazhong University of Science and Technology（华中科技大学）；Griffith University（格里菲斯大学）；Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属协和医院）；Lehigh University（理海大学）
Federated learning (FL) is a popular distributed learning framework that can reduce privacy risks by not explicitly sharing private data. However, recent works demonstrated that sharing model updates makes FL vulnerable to inference attacks. In this work, we show our key observation that the data representation leakage from gradients is the essential cause of privacy leakage in FL. We also provide an analysis of this observation to explain how the data presentation is leaked. Based on this observation, we propose a defense against model inversion attack in FL. The key idea of our defense is le…
Evaluating Selective Encryption Against Gradient Inversion Attacks
Information Networking Institute（信息网络研究所）；Carnegie Mellon University（卡内基梅隆大学）；TensorOpera AI；Department of Electrical and Computer Engineering（电气与计算机工程系）
Federated learning reduces the risk of information leakage, but remains vulnerable to attacks. We investigate how several neural network design decisions can defend against gradients inversion attacks. We show that overlapping gradients provides numerical resistance to gradient inversion on the highly vulnerable dense layer. Specifically, we propose to leverage batching to maximise mixing of gradients by choosing an appropriate loss function and drawing identical labels. We show that otherwise it is possible to directly recover all vectors in a mini-batch without any numerical optimisation due…
Extracting Spatiotemporal Data from Gradients with Large Language Models
Tokyo Institute of Technology（东京工业大学）；The University of Tokyo（东京大学）；Xidian University（西安电子科技大学）；National Institute of Information and Communications Technology（日本信息通信研究机构）；Osaka Seikei University（大阪成蹊大学）
Harvard University（哈佛大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Virginia Tech（弗吉尼亚理工大学）
This paper studies defense mechanisms against model inversion (MI) attacks -- a type of privacy attacks aimed at inferring information about the training data distribution given the access to a target machine learning model. Existing defense mechanisms rely on model-specific heuristics or noise injection. While being able to mitigate attacks, existing methods significantly hinder model performance. There remains a question of how to design a defense mechanism that is applicable to a variety of models and achieves better utility-privacy tradeoff. In this paper, we propose the Mutual Information…
GI-SMN: Gradient Inversion Attack against Federated Learning without Prior Knowledge
College of Information and Technology, Jinan University（暨南大学信息科学技术学院）；Guangdong Key Laboratory of Data Security and Privacy Protection（广东省数据安全与隐私保护重点实验室）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）
REFINE: Inversion-Free Backdoor Defense via Model Reprogramming
Zhejiang University（浙江大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链与数据安全研究院）；Nanyang Technological University（南洋理工大学）；IBM Research（IBM研究院）
Noise-Aware and Dynamically Adaptive Federated Defense Framework for SAR Image Target Recognition
Shanxi Key Laboratory of Cryptography and Data Security, School of Computer Science and Artificial Intelligence, Shanxi Normal University（山西密码与数据安全重点实验室，计算机科学与人工智能学院，山西师范大学）；School of Computer Science and Technology, Guizhou University（贵州大学计算机科学与技术学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；School of Electrical Engineering, Guangxi University（广西大学电气工程学院）；School of Computing, Engineering and Mathematical Science, La Trobe University（拉筹伯大学计算、工程与数学科学学院）；College of Cyber Security, Jinan University（济南大学网络安全学院）
MIBench: A Comprehensive Framework for Benchmarking Model Inversion Attack and Defense
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement
Zero2Text: Zero-Training Cross-Domain Inversion Attacks on Textual Embeddings
School of Computing, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院计算学院）；Graduate School of Information Security, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院信息安全研究生院）；Department of Artificial Intelligence, University of Seoul, Seoul, Republic of Korea（首尔大学人工智能系）；Department of Computer Science, University of Seoul, Seoul, Republic of Korea（首尔大学计算机科学系）
Rank Matters: Understanding and Defending Model Inversion Attacks via Low-Rank Feature Filtering
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；South China University of Technology（华南理工大学）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Random Erasing vs. Model Inversion: A Promising Defense or a False Hope?
Temasek Laboratories, Singapore University of Technology and Design（Temasek实验室，新加坡技术与设计大学）；The Queen’s University Belfast（女王大学贝尔法斯特分校）；Aarhus University（阿arhus大学）；Nanyang Technological University (NTU)（南洋理工大学（NTU）；河内 Vin 大学）；VinUniversity, Hanoi, Vietnam
Tech University of Korea（韩国科技大学）；University of Sindh（信德大学）；Munster Technological University（明斯特理工大学）；University of Johannesburg（约翰内斯堡大学）；Sheffield Hallam University（谢菲尔德哈勒姆大学）；Sungkyunkwan University（成均馆大学）
The past decade has seen a rapid adoption of Artificial Intelligence (AI), specifically the deep learning networks, in Internet of Medical Things (IoMT) ecosystem. However, it has been shown recently that the deep learning networks can be exploited by adversarial attacks that not only make IoMT vulnerable to the data theft but also to the manipulation of medical diagnosis. The existing studies consider adding noise to the raw IoMT data or model parameters which not only reduces the overall performance concerning medical inferences but also is ineffective to the likes of deep leakage from gradi…
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Tsinghua University（清华大学）；Webank（微众银行）；Shandong University（山东大学）；Hong Kong University of Science and Technology（香港科技大学）
In a vertical federated learning (VFL) scenario where features and model are split into different parties, communications of sample-specific updates are required for correct gradient calculations but can be used to deduce important sample-level label information. An immediate defense strategy is to protect sample-level messages communicated with Homomorphic Encryption (HE), and in this way only the batch-averaged local gradients are exposed to each party (termed black-boxed VFL). In this paper, we first explore the possibility of recovering labels in the vertical federated learning setting wit…
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Salesforce Research（赛富时研究院）；Microsoft Research（微软研究院）
Deepfakes pose severe threats of visual misinformation to our society. One representative deepfake application is face manipulation that modifies a victim's facial attributes in an image, e.g., changing her age or hair color. The state-of-the-art face manipulation techniques rely on Generative Adversarial Networks (GANs). In this paper, we propose the first defense system, namely UnGANable, against GAN-inversion-based face manipulation. In specific, UnGANable focuses on defending GAN inversion, an essential step for face manipulation. Its core technique is to search for alternative images (cal…
Machine learning models are vulnerable to data inference attacks, such as membership inference and model inversion attacks. In these types of breaches, an adversary attempts to infer a data record's membership in a dataset or even reconstruct this data record using a confidence score vector predicted by the target model. However, most existing defense methods only protect against membership inference attacks. Methods that can combat both types of attacks require a new model to be trained, which may not be time-efficient. In this paper, we propose a differentially private defense method that ha…
UnlearnShield: Shielding Forgotten Privacy against Unlearning Inversion
Huazhong University of Science and Technology（华中科技大学）；Institute of Guizhou Aerospace Measuring and Testing Technology（贵州航天测量测试技术研究院）；University of Technology Sydney（悉尼科技大学）；Griffith University（格里菲斯大学）
University of Science and Technology of China（中国科学技术大学）；Tencent Quantum Lab（腾讯量子实验室）
Many data mining tasks rely on graphs to model relational structures among individuals (nodes). Since relational data are often sensitive, there is an urgent need to evaluate the privacy risks in graph data. One famous privacy attack against data analysis models is the model inversion attack, which aims to infer sensitive data in the training dataset and leads to great privacy concerns. Despite its success in grid-like domains, directly applying model inversion attacks on non-grid domains such as graph leads to poor attack performance. This is mainly due to the failure to consider the unique p…
Seoul National University（首尔大学）；Ehwa University（梨花大学）
To promote secure and private artificial intelligence (SPAI), we review studies on the model security and data privacy of DNNs. Model security allows system to behave as intended without being affected by malicious external influences that can compromise its integrity and efficiency. Security attacks can be divided based on when they occur: if an attack occurs during training, it is known as a poisoning attack, and if it occurs during inference (after training) it is termed an evasion attack. Poisoning attacks compromise the training process by corrupting the data with malicious examples, whil…
