Transfer learning has been widely studied and gained increasing popularity to improve the accuracy of machine learning models by transferring some knowledge acquired in different training. However, no prior work has pointed out that transfer learning can strengthen privacy attacks on machine learning models. In this paper, we propose TransMIA (Transfer learning-based Membership Inference Attacks), which use transfer learning to perform membership inference attacks on the source model when the adversary is able to access the parameters of the transferred model. In particular, we propose a transfer shadow training technique, where an adversary employs the parameters of the transferred model to construct shadow models, to significantly improve the performance of membership inference when a limited amount of shadow training data is available to the adversary. We evaluate our attacks using two real datasets, and show that our attacks outperform the state-of-the-art that does not use our transfer shadow training technique. We also compare four combinations of the learning-based/entropy-based approach and the fine-tuning/freezing approach, all of which employ our transfer shadow training technique. Then we examine the performance of these four approaches based on the distributions of confidence values, and discuss possible countermeasures against our attacks.
Rethinking Membership Inference Attacks Against Transfer Learning
Nanyang Technological University（南洋理工大学）；Wuhan University（武汉大学）；Northeastern University（东北大学）；City University of Hong Kong（香港城市大学）；Swinburne University of Technology（斯威本科技大学）
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
While being deployed in many critical applications as core components, machine learning (ML) models are vulnerable to various security and privacy attacks. One major privacy attack in this domain is membership inference, where an adversary aims to determine whether a target data sample is part of the training set of a target ML model. So far, most of the current membership inference attacks are evaluated against ML models trained from scratch. However, real-world ML models are typically trained following the transfer learning paradigm, where a model owner takes a pretrained model learned from …
Machine Learning (ML) has made unprecedented progress in the past several decades. However, due to the memorability of the training data, ML is susceptible to various attacks, especially Membership Inference Attacks (MIAs), the objective of which is to infer the model's training data. So far, most of the membership inference attacks against ML classifiers leverage the shadow model with the same structure as the target model. However, empirical results show that these attacks can be easily mitigated if the shadow model is not clear about the network structure of the target model. In this paper,…
Membership inference attacks allow a malicious entity to predict whether a sample is used during training of a victim model or not. State-of-the-art membership inference attacks have shown to achieve good accuracy which poses a great privacy threat. However, majority of SOTA attacks require training dozens to hundreds of shadow models to accurately infer membership. This huge computation cost raises questions about practicality of these attacks on deep models. In this paper, we introduce a fundamentally different MI attack approach which obviates the need to train hundreds of shadow models. Si…
The Johns Hopkins University（约翰斯·霍普金斯大学）；The Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）；Duke University（杜克大学）
Membership inference (MI) attacks affect user privacy by inferring whether given data samples have been used to train a target learning model, e.g., a deep neural network. There are two types of MI attacks in the literature, i.e., these with and without shadow models. The success of the former heavily depends on the quality of the shadow model, i.e., the transferability between the shadow and the target; the latter, given only blackbox probing access to the target model, cannot make an effective inference of unknowns, compared with MI attacks using shadow models, due to the insufficient number…
CISPA Helmholtz Center for Information Security（亥姆霍兹信息安全中心）；Intel Labs（英特尔实验室）
Machine learning models are prone to memorizing sensitive data, making them vulnerable to membership inference attacks in which an adversary aims to infer whether an input sample was used to train the model. Over the past few years, researchers have produced many membership inference attacks and defenses. However, these attacks and defenses employ a variety of strategies and are conducted in different models and datasets. The lack of comprehensive benchmark, however, means we do not understand the strengths and weaknesses of existing attacks and defenses. We fill this gap by presenting a large…
The advent of the information age has led to the problems of information overload and unclear demands. As an information filtering system, personalized recommendation systems predict users' behavior and preference for items and improves users' information acquisition efficiency. However, recommendation systems usually use highly sensitive user data for training. In this paper, we use the latent factor model as the recommender to get the list of recommended items, and we representing users from relevant items Compared with the traditional member inference against machine learning classifiers. W…
SMI: Statistical Membership Inference for Reliable Unlearned Model Auditing
Shenzhen University of Advanced Technology（深圳先进技术大学）；Peking University（北京大学）；Xi’an Jiaotong University（西安交通大学）；Heilongjiang University（黑龙江大学）；Stevens Institute of Technology（斯蒂文斯理工学院）
When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry
Institute of Science Tokyo（东京科学大学）；Nanyang Technological University（南洋理工大学）；Tohoku University（东北大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Shandong University（山东大学）
Shandong University（山东大学）；Alibaba Group（阿里巴巴集团）；Meituan（美团）；University of Amsterdam（阿姆斯特丹大学）
Learned recommender systems may inadvertently leak information about their training data, leading to privacy violations. We investigate privacy threats faced by recommender systems through the lens of membership inference. In such attacks, an adversary aims to infer whether a user's data is used to train the target recommender. To achieve this, previous work has used a shadow recommender to derive training data for the attack model, and then predicts the membership by calculating difference vectors between users' historical interactions and recommended items. State-of-the-art methods face two …
Shandong University（山东大学）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全研究中心）
Recently, recommender systems have achieved promising performances and become one of the most widely used web applications. However, recommender systems are often trained on highly sensitive user data, thus potential data leakage from recommender systems may lead to severe privacy problems. In this paper, we make the first attempt on quantifying the privacy leakage of recommender systems through the lens of membership inference. In contrast with traditional membership inference against machine learning classifiers, our attack faces two main differences. First, our attack is on the user-level b…
University of Adelaide（阿德莱德大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织数据61研究所）
With the development of machine learning techniques, the attention of research has been moved from single-modal learning to multi-modal learning, as real-world data exist in the form of different modalities. However, multi-modal models often carry more information than single-modal models and they are usually applied in sensitive scenarios, such as medical report generation or disease identification. Compared with the existing membership inference against machine learning classifiers, we focus on the problem that the input and output of the multi-modal models are in different modalities, such …
In the text processing context, most ML models are built on word embeddings. These embeddings are themselves trained on some datasets, potentially containing sensitive data. In some cases this training is done independently, in other cases, it occurs as part of training a larger, task-specific model. In either case, it is of interest to consider membership inference attacks based on the embedding layer as a way of understanding sensitive information leakage. But, somewhat surprisingly, membership inference attacks on word embeddings and their effect in other natural language processing (NLP) t…
CISPA Helmholtz Center for Information Security（亥姆霍兹信息安全中心）；Swiss Data Science Center（瑞士数据科学中心）；ETH Zurich（苏黎世联邦理工学院）；EPFL（洛桑联邦理工学院）
Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications. However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowle…
Virginia Tech（弗吉尼亚理工大学）；Intelligent Automation, Inc.（智能自动化公司）
An over-the-air membership inference attack (MIA) is presented to leak private information from a wireless signal classifier. Machine learning (ML) provides powerful means to classify wireless signals, e.g., for PHY-layer authentication. As an adversarial machine learning attack, the MIA infers whether a signal of interest has been used in the training data of a target classifier. This private information incorporates waveform, channel, and device characteristics, and if leaked, can be exploited by an adversary to identify vulnerabilities of the underlying ML model (e.g., to infiltrate the PHY…
University of Padua（帕多瓦大学）；Delft University of Technology（代尔夫特理工大学）；Radboud University（拉德堡德大学）
Graph Neural Networks (GNNs), inspired by Convolutional Neural Networks (CNNs), aggregate the message of nodes' neighbors and structure information to acquire expressive representations of nodes for node classification, graph classification, and link prediction. Previous studies have indicated that GNNs are vulnerable to Membership Inference Attacks (MIAs), which infer whether a node is in the training data of GNNs and leak the node's private information, like the patient's disease history. The implementation of previous MIAs takes advantage of the models' probability output, which is infeasib…
Membership Inference for Contrastive Pre-training Models with Text-only PII Queries
Beijing Electronic Science and Technology Institute（北京电子科技研究所）；Institute of Artificial Intelligence, China Telecom (TeleAI)（中国电信人工智能研究院（TeleAI））；Nanyang Technological University（南洋理工大学）；Nanjing University（南京大学）；School of Computing and Information Technology, Great Bay University（广东大湾区大学计算机与信息科技学院）
Auditing Data Membership in Reinforcement Learning With Verifiable Rewards
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shanghai Jiao Tong University（上海交通大学）；Shandong University（山东大学）；Xidian University（西安电子科技大学）；Wuhan University（武汉大学）
