The prosperity of machine learning has also brought people's concerns about data privacy. Among them, inference attacks can implement privacy breaches in various MLaaS scenarios and model training/prediction phases. Specifically, inference attacks can perform privacy inference on undisclosed target training sets based on outputs of the target model, including but not limited to statistics, membership, semantics, data representation, etc. For instance, infer whether the target data has the characteristics of AIDS. In addition, the rapid development of the machine learning community in recent years, especially the surge of model types and application scenarios, has further stimulated the inference attacks' research. Thus, studying inference attacks and analyzing them in depth is urgent and significant. However, there is still a gap in the systematic discussion of inference attacks from taxonomy, global perspective, attack, and defense perspectives. This survey provides an in-depth and comprehensive inference of attacks and corresponding countermeasures in ML-as-a-service based on taxonomy and the latest researches. Without compromising researchers' intuition, we first propose the 3MP taxonomy based on the community research status, trying to normalize the confusing naming system of inference attacks. Also, we analyze the pros and cons of each type of inference attack, their workflow, countermeasure, and how they interact with other attacks. In the end, we point out several promising directions for researchers from a more comprehensive and novel perspective.
Andalusian Research Institute in Data Science and Computational Intelligence（安达卢西亚数据科学与计算智能研究院）；University of Granada（格拉纳达大学）
Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real…
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；University of Maryland, College Park（马里兰大学帕克分校）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
Deep learning has achieved overwhelming success, spanning from discriminative models to generative models. In particular, deep generative models have facilitated a new level of performance in a myriad of areas, ranging from media manipulation to sanitized dataset generation. Despite the great success, the potential risks of privacy breach caused by generative models have not been analyzed systematically. In this paper, we focus on membership inference attack against deep generative models that reveals information about the training data used for victim models. Specifically, we present the firs…
Machine learning models are shown to face a severe threat from Model Extraction Attacks, where a well-trained private model owned by a service provider can be stolen by an attacker pretending as a client. Unfortunately, prior works focus on the models trained over the Euclidean space, e.g., images and texts, while how to extract a GNN model that contains a graph structure and node features is yet to be explored. In this paper, for the first time, we comprehensively investigate and develop model extraction attacks against GNN models. We first systematically formalise the threat modelling in the…
A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models
ELLIS Alicante（欧洲学习与智能系统实验室阿利坎特分部）；Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague（捷克布拉格技术大学捷克信息学、机器人学与控制论研究所）；Delft University of Technology（代尔夫特理工大学）
Air Force Institute of Technology（空军理工学院）；Army Cyber Institute（陆军网络研究所）
The number of cyber threats against both wired and wireless computer systems and other components of the Internet of Things continues to increase annually. In this work, an algorithm selection framework is employed on the NSL-KDD data set and a novel paradigm of machine learning taxonomy is presented. The framework uses a combination of user input and meta-features to select the best algorithm to detect cyber attacks on a network. Performance is compared between a rule-of-thumb strategy and a meta-learning strategy. The framework removes the conjecture of the common trial-and-error algorithm s…
University of Maryland College Park（马里兰大学帕克分校）；University of Virginia（弗吉尼亚大学）；Cornell University（康奈尔大学）；Columbia University（哥伦比亚大学）；University of California Los Angeles（加州大学洛杉矶分校）
Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks
SBA Research（SBA研究中心）；Vienna University of Technology（维也纳工业大学）
National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所）；Sumitomo Electric Industries, Ltd.（住友电气工业株式会社）
In this article, we propose the Artificial Intelligence Security Taxonomy to systematize the knowledge of threats, vulnerabilities, and security controls of machine-learning-based (ML-based) systems. We first classify the damage caused by attacks against ML-based systems, define ML-specific security, and discuss its characteristics. Next, we enumerate all relevant assets and stakeholders and provide a general taxonomy for ML-specific threats. Then, we collect a wide range of security controls against ML-specific threats through an extensive review of recent literature. Finally, we classify the…
Federated learning (FL) enables the training of models among distributed clients without compromising the privacy of training datasets, while the invisibility of clients datasets and the training process poses a variety of security threats. This survey provides the taxonomy of poisoning attacks and experimental evaluation to discuss the need for robust FL.
Machine learning models have made many decision support systems to be faster, more accurate, and more efficient. However, applications of machine learning in network security face a more disproportionate threat of active adversarial attacks compared to other domains. This is because machine learning applications in network security such as malware detection, intrusion detection, and spam filtering are by themselves adversarial in nature. In what could be considered an arm's race between attackers and defenders, adversaries constantly probe machine learning systems with inputs that are explicit…
NVIDIA（英伟达）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations. Research explores different approaches to improve ML dependability by proposing new models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. However, there is a missing connection between ongoing ML research and well-established safety principles. In this paper, we present…
Unlike the white-box counterparts that are widely studied and readily accessible, adversarial examples in black-box settings are generally more Herculean on account of the difficulty of estimating gradients. Many methods achieve the task by issuing numerous queries to target classification systems, which makes the whole procedure costly and suspicious to the systems. In this paper, we aim at reducing the query complexity of black-box attacks in this category. We propose to exploit gradients of a few reference models which arguably span some promising search subspaces. Experimental results show…
University of Toronto（多伦多大学）；Electrical and Computer Engineering（电子与计算机工程（此处为院系，按要求不提取，但原文中该部分是否为机构？根据规则，机构粒度提取到大学/学院/公司/研究院级别即可，不要细拆到系等，因此仅保留“University of Toronto”））
With the rise of artificial intelligence and machine learning in modern computing, one of the major concerns regarding such techniques is to provide privacy and security against adversaries. We present this survey paper to cover the most representative papers in poisoning attacks against supervised machine learning models. We first provide a taxonomy to categorize existing studies and then present detailed summaries for selected papers. We summarize and compare the methodology and limitations of existing literature. We conclude this paper with potential improvements and future directions to fu…
SoK: Reconstruction Attacks on Synthetic Tabular Data (Insights from Winning the NIST CRC)
School of Engineering and Technology, University of Washington Tacoma（华盛顿大学塔科姆分校工程与技术学院）；Department of Mathematics, Computer Science, and Statistics, Ghent University（根特大学数学、计算机科学与统计学系）
Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Beijing University of Posts and Telecommunications（北京邮电大学）；University at Buffalo, State University of New York（纽约州立大学布法罗分校）
A Survey on Model Extraction Attacks and Defenses for Large Language Models
University of Notre Dame（圣约翰大学）；Florida State University（佛罗里达州立大学）；Northwestern University（西北大学）；Duke University（杜克大学）；University of Southern California（南加州大学）
As machine learning expanding application, there are more and more unignorable privacy and safety issues. Especially inference attacks against Machine Learning models allow adversaries to infer sensitive information about the target model, such as training data, model parameters, etc. Inference attacks can lead to serious consequences, including violating individuals privacy, compromising the intellectual property of the owner of the machine learning model. As far as concerned, researchers have studied and analyzed in depth several types of inference attacks, albeit in isolation, but there is …
The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong Polytechnic University Shenzhen Research Institute（香港理工大学深圳研究院）；JD Explore Academy, JD.com（京东探索研究院）
Recent studies have shown that the training samples can be recovered from gradients, which are called Gradient Inversion (GradInv) attacks. However, there remains a lack of extensive surveys covering recent advances and thorough analysis of this issue. In this paper, we present a comprehensive survey on GradInv, aiming to summarize the cutting-edge research and broaden the horizons for different domains. Firstly, we propose a taxonomy of GradInv attacks by characterizing existing attacks into two paradigms: iteration- and recursion-based attacks. In particular, we dig out some critical ingredi…
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Nanjing University of Science and Technology（南京理工大学）；Xidian University（西安电子科技大学）；CSIRO（联邦科学与工业研究组织）；University of Western Australia（西澳大学）；University of Technology Sydney（悉尼科技大学）
University of Florida（佛罗里达大学）；University of Toronto（多伦多大学）
Speech and speaker recognition systems are employed in a variety of applications, from personal assistants to telephony surveillance and biometric authentication. The wide deployment of these systems has been made possible by the improved accuracy in neural networks. Like other systems based on neural networks, recent research has demonstrated that speech and speaker recognition systems are vulnerable to attacks using manipulated inputs. However, as we demonstrate in this paper, the end-to-end architecture of speech and speaker systems and the nature of their inputs make attacks and defenses a…
Westlake University（西湖大学）；Westlake Institute for Advanced Study（西湖高等研究院）；University of Chinese Academy of Sciences（中国科学院大学）；George Mason University（乔治梅森大学）；Chinese Academy of Sciences（中国科学院）
Pretrained Language Models (PLMs) such as BERT have revolutionized the landscape of Natural Language Processing (NLP). Inspired by their proliferation, tremendous efforts have been devoted to Pretrained Graph Models (PGMs). Owing to the powerful model architectures of PGMs, abundant knowledge from massive labeled and unlabeled graph data can be captured. The knowledge implicitly encoded in model parameters can benefit various downstream tasks and help to alleviate several fundamental issues of learning on graphs. In this paper, we provide the first comprehensive survey for PGMs. We firstly pre…
