Recently, researchers have started decomposing deep neural network models according to their semantics or functions. Recent work has shown the effectiveness of decomposed functional blocks for defending adversarial attacks, which add small input perturbation to the input image to fool the DNN models. This work proposes a profiling-based method to decompose the DNN models to different functional blocks, which lead to the effective path as a new approach to exploring DNNs' internal organization. Specifically, the per-image effective path can be aggregated to the class-level effective path, through which we observe that adversarial images activate effective path different from normal images. We propose an effective path similarity-based method to detect adversarial images with an interpretable model, which achieve better accuracy and broader applicability than the state-of-the-art technique.
The privacy of machine learning models has become a significant concern in many emerging Machine-Learning-as-a-Service applications, where prediction services based on well-trained models are offered to users via pay-per-query. The lack of a defense mechanism can impose a high risk on the privacy of the server's model since an adversary could efficiently steal the model by querying only a few `good' data points. The interplay between a server's defense and an adversary's attack inevitably leads to an arms race dilemma, as commonly seen in Adversarial Machine Learning. To study the fundamental …
Model extraction attacks aim to duplicate a machine learning model through query access to a target model. Early studies mainly focus on discriminative models. Despite the success, model extraction attacks against generative models are less well explored. In this paper, we systematically study the feasibility of model extraction attacks against generative adversarial networks (GANs). Specifically, we first define accuracy and fidelity on model extraction attacks against GANs. Then we study model extraction attacks against GANs from the perspective of accuracy extraction and fidelity extraction…
Sun Yat-sen University（中山大学）；Shuguang Hospital, Shanghai University of Traditional Chinese Medicine（上海中医药大学附属曙光医院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
As the COVID-19 pandemic puts pressure on healthcare systems worldwide, the computed tomography image based AI diagnostic system has become a sustainable solution for early diagnosis. However, the model-wise vulnerability under adversarial perturbation hinders its deployment in practical situation. The existing adversarial training strategies are difficult to generalized into medical imaging field challenged by complex medical texture features. To overcome this challenge, we propose a Contour Attention Preserving (CAP) method based on lung cavity edge extraction. The contour prior features are…
A Survey on Model Extraction Attacks and Defenses for Large Language Models
University of Notre Dame（圣约翰大学）；Florida State University（佛罗里达州立大学）；Northwestern University（西北大学）；Duke University（杜克大学）；University of Southern California（南加州大学）
George Mason University（乔治梅森大学）；Drexel University（德雷塞尔大学）；University of California, Davis（加州大学戴维斯分校）
This paper presents a novel model training solution, denoted as Adaptive-Gravity, for enhancing the robustness of deep neural network classifiers against adversarial examples. We conceptualize the model parameters/features associated with each class as a mass characterized by its centroid location and the spread (standard deviation of the distance) of features around the centroid. We use the centroid associated with each cluster to derive an anti-gravity force that pushes the centroids of different classes away from one another during network training. Then we customized an objective function …
Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）；Japan Science and Technology Agency（日本科学技术振兴机构）
Model extraction attacks have become serious issues for service providers using machine learning. We consider an adversarial setting to prevent model extraction under the assumption that attackers will make their best guess on the service provider's model using query accesses, and propose to build a surrogate model that significantly keeps away the predictions of the attacker's model from those of the true model. We formulate the problem as a non-convex constrained bilevel optimization problem and show that for kernel models, it can be transformed into a non-convex 1-quadratically constrained …
Artificial neural networks (ANNs) have gained significant popularity in the last decade for solving narrow AI problems in domains such as healthcare, transportation, and defense. As ANNs become more ubiquitous, it is imperative to understand their associated safety, security, and privacy vulnerabilities. Recently, it has been shown that ANNs are susceptible to a number of adversarial evasion attacks--inputs that cause the ANN to make high-confidence misclassifications despite being almost indistinguishable from the data used to train and test the network. This work explores to what degree find…
A Survey of Model Extraction Attacks and Defenses in Distributed Computing Environments
University of Notre Dame（圣母大学）；Florida State University（佛罗里达州立大学）；Northwestern University（西北大学）；Duke University（杜克大学）；University of Southern California（南加州大学）
Alert-ME: An Explainability-Driven Defense Against Adversarial Examples in Transformer-Based Text Classification
School of Computer Science, The University of Adelaide（阿德莱德大学计算机科学学院）；CREST - The Centre for Research on Engineering Software Technologies（工程软件技术研究中心）；Department of Computer Science and Software Engineering, The University of Western Australia（西澳大学计算机科学与软件工程学院）
National Institute of Technology Karnataka（卡纳塔克邦国家理工学院）；Indian Institute of Science（印度科学学院）
Many commercial and forensic applications of speech demand the extraction of information about the speaker characteristics, which falls into the broad category of speaker profiling. The speaker characteristics needed for profiling include physical traits of the speaker like height, age, and gender of the speaker along with the native language of the speaker. Many of the datasets available have only partial information for speaker profiling. In this paper, we attempt to overcome this limitation by developing a new dataset which has speech data from five different Indian languages along with Eng…
GraphRank Pro+: Advancing Talent Analytics Through Knowledge Graphs and Sentiment-Enhanced Skill Profiling
UCEK, JNTU Kakinada（贾夫纳大学工程学院，贾夫纳印度理工学院卡基纳达分校）；Brane Enterprises（布兰恩企业）；School of EECS, College of Engineering and Mines, University of North Dakota（北达科他大学工程与矿业学院电子工程与计算机科学学院）
A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives
University of Notre Dame（诺丁汉大学）；Florida State University（佛罗里达州立大学）；Northwestern University（西北大学）；Duke University（杜克大学）；University of Southern California（南加州大学）
CITED: A Decision Boundary-Aware Signature for GNNs Towards Model Extraction Defense
Department of Computer Science, Florida State University, Tallahassee, Florida, United States（佛罗里达州立大学计算机科学系）；Department of Mathematics, University of Wisconsin, Madison, Wisconsin, United States（威斯康星大学麦迪逊分校数学系）
Xidian University（西安电子科技大学）；Purdue University（普渡大学）；Chinese Academic of Sciences（中国科学院）
Public intelligent services enabled by machine learning algorithms are vulnerable to model extraction attacks that can steal confidential information of the learning models through public queries. Though there are some protection options such as differential privacy (DP) and monitoring, which are considered promising techniques to mitigate this attack, we still find that the vulnerability persists. In this paper, we propose an adaptive query-flooding parameter duplication (QPD) attack. The adversary can infer the model information with black-box access and no prior knowledge of any model param…
Pareto-Secure Machine Learning (PSML): Fingerprinting and Securing Inference Serving Systems
Georgia Institute of Technology（佐治亚理工学院）；University of California, Riverside（加州大学河滨分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Virginia（弗吉尼亚大学）；George Washington University（乔治华盛顿大学）
With the recent advancements in machine learning theory, many commercial embedded micro-processors use neural network models for a variety of signal processing applications. However, their associated side-channel security vulnerabilities pose a major concern. There have been several proof-of-concept attacks demonstrating the extraction of their model parameters and input data. But, many of these attacks involve specific assumptions, have limited applicability, or pose huge overheads to the attacker. In this work, we study the side-channel vulnerabilities of embedded neural network implementati…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Stanford University（斯坦福大学）
In model extraction attacks, adversaries can steal a machine learning model exposed via a public API by repeatedly querying it and adjusting their own model based on obtained predictions. To prevent model stealing, existing defenses focus on detecting malicious queries, truncating, or distorting outputs, thus necessarily introducing a tradeoff between robustness and model utility for legitimate users. Instead, we propose to impede model extraction by requiring users to complete a proof-of-work before they can read the model's predictions. This deters attackers by greatly increasing (even up to…
This paper proposes a new defense against neural network backdooring attacks that are maliciously trained to mispredict in the presence of attacker-chosen triggers. Our defense is based on the intuition that the feature extraction layers of a backdoored network embed new features to detect the presence of a trigger and the subsequent classification layers learn to mispredict when triggers are detected. Therefore, to detect backdoors, the proposed defense uses two synergistic anomaly detectors trained on clean validation data: the first is a novelty detector that checks for anomalous features, …
Aalto University（阿尔托大学）；University of Waterloo（滑铁卢大学）；University of Helsinki（赫尔辛基大学）
Machine learning models are typically made available to potential client users via inference APIs. Model extraction attacks occur when a malicious client uses information gleaned from queries to the inference API of a victim model $F_V$ to build a surrogate model $F_A$ with comparable functionality. Recent research has shown successful model extraction of image classification, and natural language processing models. In this paper, we show the first model extraction attack against real-world generative adversarial network (GAN) image translation models. We present a framework for conducting suc…
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）
Self-Supervised Learning (SSL) is an increasingly popular ML paradigm that trains models to transform complex inputs into representations without relying on explicit labels. These representations encode similarity structures that enable efficient learning of multiple downstream tasks. Recently, ML-as-a-Service providers have commenced offering trained SSL models over inference APIs, which transform user inputs into useful representations for a fee. However, the high cost involved to train these models and their exposure over APIs both make black-box extraction a realistic security threat. We t…
We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model. Assuming that both the adversary and victim model fine-tune a large pretrained language model such as BERT (Devlin et al. 2019), we show that the adversary does not need any real training data to successfully mount the attack. In fact, the attacker need not even use grammatical or semantically meaningful queries: we show that random sequences of words coupled with task-specific heuristics form effective queri…
Structure-Guided Visual Perturbation Neutralization for LVLMs
Beijing University of Posts and Telecommunications（北京邮电大学）；University of Science and Technology of China（中国科学技术大学）；JIUTIAN Research（中移九天）；Nanyang Technological University（南洋理工大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Adversarial Vulnerability Transcends Computational Paradigms: Feature Engineering Provides No Defense Against Neural Adversarial Transfer
1 Department of Information \& Computer Science, KFUPM, Dhahran, Saudi Arabia；2 Department of Computer Engineering, KFUPM, Dhahran, Saudi Arabia；3 Interdisciplinary Research Center for Finance
CREDIT: Certified Ownership Verification of Deep Neural Networks Against Model Extraction Attacks
Department of Computer Science, Florida State University, Tallahassee, Florida, United States（佛罗里达州立大学计算机科学系）；Department of Mathematics, University of Wisconsin, Madison, Wisconsin, United States（威斯康星大学麦迪逊分校数学系）；Department of Computer Science, Duke University, Durham, North Carolina, United States（杜克大学计算机科学系）；Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States（北卡罗来纳大学教堂山分校统计与运筹学系）
