Self-supervised representation learning techniques have been developing rapidly to make full use of unlabeled images. They encode images into rich features that are oblivious to downstream tasks. Behind their revolutionary representation power, the requirements for dedicated model designs and a massive amount of computation resources expose image encoders to the risks of potential model stealing attacks - a cheap way to mimic the well-trained encoder performance while circumventing the demanding requirements. Yet conventional attacks only target supervised classifiers given their predicted labels and/or posteriors, which leaves the vulnerability of unsupervised encoders unexplored.
In this paper, we first instantiate the conventional stealing attacks against encoders and demonstrate their severer vulnerability compared with downstream classifiers. To better leverage the rich representation of encoders, we further propose Cont-Steal, a contrastive-learning-based attack, and validate its improved stealing effectiveness in various experiment settings. As a takeaway, we appeal to our community's attention to the intellectual property protection of representation learning techniques, especially to the defenses against encoder stealing attacks like ours.
Pre-trained encoders are general-purpose feature extractors that can be used for many downstream tasks. Recent progress in self-supervised learning can pre-train highly effective encoders using a large volume of unlabeled data, leading to the emerging encoder as a service (EaaS). A pre-trained encoder may be deemed confidential because its training requires lots of data and computation resources as well as its public release may facilitate misuse of AI, e.g., for deepfakes generation. In this paper, we propose the first attack called StolenEncoder to steal pre-trained image encoders. We evalua…
Tsinghua University（清华大学）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
Self-supervised learning is an emerging machine learning paradigm. Compared to supervised learning which leverages high-quality labeled datasets, self-supervised learning relies on unlabeled datasets to pre-train powerful encoders which can then be treated as feature extractors for various downstream tasks. The huge amount of data and computational resources consumption makes the encoders themselves become the valuable intellectual property of the model owner. Recent research has shown that the machine learning model's copyright is threatened by model stealing attacks, which aim to train a sur…
Graduate School of Computer Science, Saarland University（萨尔大学计算机科学研究生院）；Bosch Center for Artificial Intelligence, Robert Bosch GmbH（罗伯特·博世有限公司博世人工智能中心）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
Model-Guardian: Protecting against Data-Free Model Stealing Using Gradient Representations and Deceptive Predictions
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；State Key Laboratory of Cyberspace Security Defense（网络空间安全防御国家重点实验室）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Norton Research Group（诺顿研究组）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；INRIA（法国国家信息与自动化研究所）
Many real-world data come in the form of graphs. Graph neural networks (GNNs), a new family of machine learning (ML) models, have been proposed to fully leverage graph data to build powerful applications. In particular, the inductive GNNs, which can generalize to unseen data, become mainstream in this direction. Machine learning models have shown great potential in various tasks and have been deployed in many real-world scenarios. To train a good model, a large amount of data as well as computational resources are needed, leading to valuable intellectual property. Previous research has shown t…
Hyperparameters are critical in machine learning, as different hyperparameters often result in models with significantly different performance. Hyperparameters may be deemed confidential because of their commercial value and the confidentiality of the proprietary algorithms that the learner uses to learn them. In this work, we propose attacks on stealing the hyperparameters that are learned by a learner. We call our attacks hyperparameter stealing attacks. Our attacks are applicable to a variety of popular machine learning algorithms such as ridge regression, logistic regression, support vecto…
CISPA Helmholtz Center for Information Security（CISPA 亥姆霍兹信息安全中心）；Duke University（杜克大学）
Graph data, such as chemical networks and social networks, may be deemed confidential/private because the data owner often spends lots of resources collecting the data or the data contains sensitive information, e.g., social relationships. Recently, neural networks were extended to graph data, which are known as graph neural networks (GNNs). Due to their superior performance, GNNs have many applications, such as healthcare analytics, recommender systems, and fraud detection. In this work, we propose the first attacks to steal a graph from the outputs of a GNN model that is trained on the graph…
I Stolenly Swear That I Am Up to (No) Good: Design and Evaluation of Model Stealing Attacks
Christian Doppler Laboratory for Assurance；Transparency in Software Protection, University of Vienna Vienna Austria；SBA Research \& TU Wien Vienna Austria；IBM Research Zurich Zurich Switzerland；Institute of Information Systems Engineering, TU Wien Vienna Austria；Transparency in Software Protection, University of Vienna；SBA Research \& TU Wien；IBM Research Zurich；Institute of Information Systems Engineering, TU Wien
Deep Neural Networks (DNNs) are susceptible to model stealing attacks, which allows a data-limited adversary with no knowledge of the training dataset to clone the functionality of a target model, just by using black-box query access. Such attacks are typically carried out by querying the target model using inputs that are synthetically generated or sampled from a surrogate dataset to construct a labeled dataset. The adversary can use this labeled dataset to train a clone model, which achieves a classification accuracy comparable to that of the target model. We propose "Adaptive Misinformation…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Saarland Informatics Campus, Germany（德国萨尔州信息学园区）
High-performance Deep Neural Networks (DNNs) are increasingly deployed in many real-world applications e.g., cloud prediction APIs. Recent advances in model functionality stealing attacks via black-box access (i.e., inputs in, predictions out) threaten the business model of such applications, which require a lot of time, money, and effort to develop. Existing defenses take a passive role against stealing attacks, such as by truncating predicted information. We find such passive defenses ineffective against DNN stealing attacks. In this paper, we propose the first defense which actively perturb…
Swinburne University of Technology（斯威本科技大学）；Deakin University（迪肯大学）
Stealing attack against controlled information, along with the increasing number of information leakage incidents, has become an emerging cyber security threat in recent years. Due to the booming development and deployment of advanced analytics solutions, novel stealing attacks utilize machine learning (ML) algorithms to achieve high success rate and cause a lot of damage. Detecting and defending against such attacks is challenging and urgent so that governments, organizations, and individuals should attach great importance to the ML-based stealing attacks. This survey presents the recent adva…
In privacy-preserving machine learning, it is common that the owner of the learned model does not have any physical access to the data. Instead, only a secured remote access to a data lake is granted to the model owner without any ability to retrieve data from the data lake. Yet, the model owner may want to export the trained model periodically from the remote repository and a question arises whether this may cause is a risk of data leakage. In this paper, we introduce the concept of data stealing attack during the export of neural networks. It consists in hiding some information in the export…
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；NetApp（NetApp公司）；University of Virginia（弗吉尼亚大学）
Building advanced machine learning (ML) models requires expert knowledge and many trials to discover the best architecture and hyperparameter settings. Previous work demonstrates that model information can be leveraged to assist other attacks, such as membership inference, generating adversarial examples. Therefore, such information, e.g., hyperparameters, should be kept confidential. It is well known that an adversary can leverage a target ML model's output to steal the model's information. In this paper, we discover a new side channel for model information stealing attacks, i.e., models' sci…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；CISPA Helmholtz Center i.G.（CISPA亥姆霍兹信息安全中心）
Machine Learning (ML) models are increasingly deployed in the wild to perform a wide range of tasks. In this work, we ask to what extent can an adversary steal functionality of such "victim" models based solely on blackbox interactions: image in, predictions out. In contrast to prior work, we present an adversary lacking knowledge of train/test data used by the model, its internals, and semantics over model outputs. We formulate model functionality stealing as a two-step approach: (i) querying a set of input images to the blackbox model to obtain predictions; and (ii) training a "knockoff" wit…
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Obtaining a well-trained model involves expensive data collection and training procedures, therefore the model is a valuable intellectual property. Recent studies revealed that adversaries can `steal' deployed models even when they have no training samples and can not get access to the model parameters or structures. Currently, there were some defense methods to alleviate this threat, mostly by increasing the cost of model stealing. In this paper, we explore the defense from another angle by verifying whether a suspicious model contains the knowledge of defender-specified \emph{external featur…
Link Stealing Attacks Against Inductive Graph Neural Networks
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全研究中心）；Hong Kong University of Science and Technology（香港科技大学）；University of Birmingham（伯明翰大学）；University of Lausanne（洛桑大学）；Duke University（杜克大学）
BESA: Boosting Encoder Stealing Attack with Perturbation Recovery
School of Cyberspace Science and Technology, Beijing Institute of Technology（网络安全科学与技术学院，北京理工大学）；Beijing Institute of Technology（北京理工大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Prompt Pirates Need a Map: Stealing Seeds helps Stealing Prompts
Visual Computing and Artificial Intelligence, Kiel University, Germany（视觉计算与人工智能，基尔大学，德国）；Artificial Intelligence, Kiel University, Germany（人工智能，基尔大学，德国）
Isolation and Induction: Training Robust Deep Neural Networks against Model Stealing Attacks
Beihang University（北京航空航天大学）；Institute of Dataspace（数据空间研究院）；Chinese Academy of Sciences（中国科学院）；Sensetime Group Limited（商汤集团有限公司）；Zhongguancun Laboratory（中关村实验室）
Michigan Technological University（密歇根理工大学）；American University（美国大学）；Cognization Lab（认知实验室）；University of Florida（佛罗里达大学）
Deep neural networks (DNNs) have become the essential components for various commercialized machine learning services, such as Machine Learning as a Service (MLaaS). Recent studies show that machine learning services face severe privacy threats - well-trained DNNs owned by MLaaS providers can be stolen through public APIs, namely model stealing attacks. However, most existing works undervalued the impact of such attacks, where a successful attack has to acquire confidential training data or auxiliary data regarding the victim DNN. In this paper, we propose ES Attack, a novel model stealing att…
United Nations University（联合国大学）；The University of Tokyo（东京大学）
An attack on deep learning systems where intelligent machines collaborate to solve problems could cause a node in the network to make a mistake on a critical judgment. At the same time, the security and privacy concerns of AI have galvanized the attention of experts from multiple disciplines. In this research, we successfully mounted adversarial attacks on a federated learning (FL) environment using three different datasets. The attacks leveraged generative adversarial networks (GANs) to affect the learning process and strive to reconstruct the private data of users by learning hidden features…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
An off-the-shelf model as a commercial service could be stolen by model stealing attacks, posing great threats to the rights of the model owner. Model fingerprinting aims to verify whether a suspect model is stolen from the victim model, which gains more and more attention nowadays. Previous methods always leverage the transferable adversarial examples as the model fingerprint, which is sensitive to adversarial defense or transfer learning scenarios. To address this issue, we consider the pairwise relationship between samples instead and propose a novel yet simple model stealing detection meth…
Hyperdimensional Computing (HDC) is facing infringement issues due to straightforward computations. This work, for the first time, raises a critical vulnerability of HDC, an attacker can reverse engineer the entire model, only requiring the unindexed hypervector memory. To mitigate this attack, we propose a defense strategy, namely HDLock, which significantly increases the reasoning cost of encoding. Specifically, HDLock adds extra feature hypervector combination and permutation in the encoding module. Compared to the standard HDC model, a two-layer-key HDLock can increase the adversarial reas…
We performed the first systematic study of a new attack on Ethereum that steals cryptocurrencies. The attack is due to the unprotected JSON-RPC endpoints existed in Ethereum nodes that could be exploited by attackers to transfer the Ether and ERC20 tokens to attackers-controlled accounts. This study aims to shed light on the attack, including malicious behaviors and profits of attackers. Specifically, we first designed and implemented a honeypot that could capture real attacks in the wild. We then deployed the honeypot and reported results of the collected data in a period of six months. In to…
Considering computer systems, security is the major concern with usability. Security policies need to be developed to protect information from unauthorized access. Passwords and secrete codes used between users and information systems for secure user authentication with the system. Playing a vital role in security, easily guessed passwords are links to vulnerability. They allow invader to put system resources significantly closer to access them, other accounts on nearby machines and possibly even administrative privileges with different threats and vulnerabilities (e.g., phishing, key logging …
Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment
Bosch Center for Artificial Intelligence（博世人工智能中心）；Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Saarland University（萨尔大学）；University of Strasbourg（斯特拉斯堡大学）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
Training deep neural networks often forces users to work in a distributed or outsourced setting, accompanied with privacy concerns. Split learning aims to address this concern by distributing the model among a client and a server. The scheme supposedly provides privacy, since the server cannot see the clients' models and inputs. We show that this is not true via two novel attacks. (1) We show that an honest-but-curious split learning server, equipped only with the knowledge of the client neural network architecture, can recover the input samples and obtain a functionally similar model to the c…
Machine learning models deployed as a service (MLaaS) are susceptible to model stealing attacks, where an adversary attempts to steal the model within a restricted access framework. While existing attacks demonstrate near-perfect clone-model performance using softmax predictions of the classification network, most of the APIs allow access to only the top-1 labels. In this work, we show that it is indeed possible to steal Machine Learning models by accessing only top-1 predictions (Hard Label setting) as well, without access to model gradients (Black-Box setting) or even the training dataset (D…
