NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training
Massachusetts Institute of Technology (MIT)（麻省理工学院）；Maynooth University（梅努斯大学）
Adam Yala, Homa Esfahanizadeh, Rafael G. L. D' Oliveira, Ken R. Duffy, Manya Ghobadi, Tommi S. Jaakkola, Vinod Vaikuntanathan, Regina Barzilay, Muriel Medard
Balancing the needs of data privacy and predictive utility is a central challenge for machine learning in healthcare. In particular, privacy concerns have led to a dearth of public datasets, complicated the construction of multi-hospital cohorts and limited the utilization of external machine learning resources. To remedy this, new methods are required to enable data owners, such as hospitals, to share their datasets publicly, while preserving both patient privacy and modeling utility. We propose NeuraCrypt, a private encoding scheme based on random deep neural networks. NeuraCrypt encodes raw patient data using a randomly constructed neural network known only to the data-owner, and publishes both the encoded data and associated labels publicly. From a theoretical perspective, we demonstrate that sampling from a sufficiently rich family of encoding functions offers a well-defined and meaningful notion of privacy against a computationally unbounded adversary with full knowledge of the underlying data-distribution. We propose to approximate this family of encoding functions through random deep neural networks. Empirically, we demonstrate the robustness of our encoding to a suite of adversarial attacks and show that NeuraCrypt achieves competitive accuracy to non-private baselines on a variety of x-ray tasks. Moreover, we demonstrate that multiple hospitals, using independent private encoders, can collaborate to train improved x-ray models. Finally, we release a challenge dataset to encourage the development of new attacks on NeuraCrypt.
NeuraCrypt (Yara et al. arXiv 2021) is an algorithm that converts a sensitive dataset to an encoded dataset so that (1) it is still possible to train machine learning models on the encoded data, but (2) an adversary who has access only to the encoded dataset can not learn much about the original sensitive dataset. We break NeuraCrypt privacy claims, by perfectly solving the authors' public challenge, and by showing that NeuraCrypt does not satisfy the formal privacy definitions posed in the original paper. Our attack consists of a series of boosting steps that, coupled with various design flaw…
In recent years, with the development of cloud computing platforms, privacy-preserving methods for deep learning have become an urgent problem. NeuraCrypt is a private random neural network for privacy-preserving that allows data owners to encrypt the medical data before the data uploading, and data owners can train and then test their models in a cloud server with the encrypted data directly. However, we point out that the performance of NeuraCrypt is heavily degraded when using color images. In this paper, we propose a Color-NeuraCrypt to solve this problem. Experiment results show that our …
In this paper, we present a novel insider attack called Matryoshka, which employs an irrelevant scheduled-to-publish DNN model as a carrier model for covert transmission of multiple secret models which memorize the functionality of private ML data stored in local data centers. Instead of treating the parameters of the carrier model as bit strings and applying conventional steganography, we devise a novel parameter sharing approach which exploits the learning capacity of the carrier model for information hiding. Matryoshka simultaneously achieves: (i) High Capacity -- With almost no utility los…
Pointing the Way, Hiding the Destination: Practical Private Dense Retrieval at Scale
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Tsinghua University（清华大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；The Chinese University of Hong Kong（香港中文大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
The amount of personal data collected in our everyday interactions with connected devices offers great opportunities for innovative services fueled by machine learning, as well as raises serious concerns for the privacy of individuals. In this paper, we propose a massively distributed protocol for a large set of users to privately compute averages over their joint data, which can then be used to learn predictive models. Our protocol can find a solution of arbitrary accuracy, does not rely on a third party and preserves the privacy of users throughout the execution in both the honest-but-curiou…
This paper proposes a fully homomorphic encryption encapsulated difference expansion (FHEE-DE) scheme for reversible data hiding in encrypted domain (RDH-ED). In the proposed scheme, we use key-switching and bootstrapping techniques to control the ciphertext extension and decryption failure. To realize the data extraction directly from the encrypted domain without the private key, a key-switching based least-significant-bit (KS-LSB) data hiding method has been designed. In application, the user first encrypts the plaintext and uploads ciphertext to the server. Then the server performs data hid…
Princeton University（普林斯顿大学）；Institute for Advanced Study（高等研究院）
How can multiple distributed entities collaboratively train a shared deep net on their private data while preserving privacy? This paper introduces InstaHide, a simple encryption of training images, which can be plugged into existing distributed deep learning pipelines. The encryption is efficient and applying it during training has minor effect on test accuracy. InstaHide encrypts each training image with a "one-time secret key" which consists of mixing a number of randomly chosen images and applying a random pixel-wise mask. Other contributions of this paper include: (a) Using a large public…
In today's world the art of sending & displaying the hidden information especially in public places, has received more attention and faced many challenges. Therefore, different methods have been proposed so far for hiding information in different cover media. In this paper a method for hiding of information on the billboard display is presented. It is well known that encryption provides secure channels for communicating entities. However, due to lack of covertness on these channels, an eavesdropper can identify encrypted streams through statistical tests and capture them for further cryptanaly…
In this paper, we have introduced a new type of leakage associated with modern encrypted databases called update pattern leakage. We formalize the definition and security model of DP-Sync with DP update patterns. We also proposed the framework DP-Sync, which extends existing encrypted database schemes to DP-Sync with DP update patterns. DP-Sync guarantees that the entire data update history over the outsourced data structure is protected by differential privacy. This is achieved by imposing differentially-private strategies that dictate the data owner's synchronization of local~data.
Differentially Private Secure Multiplication: Hiding Information in the Rubble of Noise
Georgia Institute of Technology（佐治亚理工学院）；Pennsylvania State University（宾夕法尼亚州立大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Harvard University（哈佛大学）
Privacy of the outsourced data is one of the major challenge.Insecurity of the network environment and untrustworthiness of the service providers are obstacles of making the database as a service.Collection and storage of personally identifiable information is a major privacy concern.On-line public databases and resources pose a significant risk to user privacy, since a malicious database owner may monitor user queries and infer useful information about the customer.The challenge in data privacy is to share data with third-party and at the same time securing the valuable information from unaut…
We present an algorithm for the recovery of a matrix $\mathbb{M}$ % (non-singular $\in $ $\mathbb{C}^{N\times N}$) by only being aware of two of its powers, $\mathbb{M}_{k_{1}}:=\mathbb{M}^{k_{1}}$ and $\mathbb{M}% _{k_{2}}:=\mathbb{M}^{k_{2}}$ ($k_{1}>k_{2}$) whose exponents are positive coprime numbers. The knowledge of the exponents is the key to retrieve matrix $\mathbb{M}$ out from the two matrices $\mathbb{M}_{k_{i}}$. The procedure combines products and inversions of matrices, and a few computational steps are needed to get $\mathbb{M}$, almost independently of the exponents magnitudes.…
Recent work of Erlingsson, Feldman, Mironov, Raghunathan, Talwar, and Thakurta [EFMRTT19] demonstrates that random shuffling amplifies differential privacy guarantees of locally randomized data. Such amplification implies substantially stronger privacy guarantees for systems in which data is contributed anonymously [BEMMRLRKTS17] and has lead to significant interest in the shuffle model of privacy [CSUZZ19; EFMRTT19]. We show that random shuffling of $n$ data records that are input to $\varepsilon_0$-differentially private local randomizers results in an $(O((1-e^{-\varepsilon_0})\sqrt{\frac{e…
Disentangling private classes through regularization
LTCI, Telecom Paris, Institut Polytechnique de Paris（巴黎理工学院电信巴黎分校LTCI实验室）；Mykolas Romeris University（米科拉斯·罗梅里斯大学）；University of Bologna（博洛尼亚大学）；University of Turin（都灵大学）
Motivation: Researchers need a rich trove of genomic datasets that they can leverage to gain a better understanding of the genetic basis of the human genome and identify associations between phenotypes and specific parts of DNA. However, sharing genomic datasets that include sensitive genetic or medical information of individuals can lead to serious privacy-related consequences if data lands in the wrong hands. Restricting access to genomic datasets is one solution, but this greatly reduces their usefulness for research purposes. To allow sharing of genomic datasets while addressing these priv…
In a beyond-5G (B5G) scenario, we consider a virtual private mobile network (VPMN), i.e., a set of user equipments (UEs) directly communicating in a device-to-device (D2D) fashion, and connected to the cellular network by multiple gateways. The purpose of the VPMN is to hide the position of the VPMN UEs to the mobile network operator (MNO). We investigate the design and performance of packet routing inside the VPMN. First, we note that the routing that maximizes the rate between the VPMN and the cellular network leads to an unbalanced use of the gateways by each UE. In turn, this reveals infor…
Personal data has value to both its owner and to institutions who would like to analyze it. Privacy mechanisms protect the owner's data while releasing to analysts noisy versions of aggregate query results. But such strict protections of individual's data have not yet found wide use in practice. Instead, Internet companies, for example, commonly provide free services in return for valuable sensitive information from users, which they exploit and sometimes sell to third parties. As the awareness of the value of the personal data increases, so has the drive to compensate the end user for her pri…
Private and Robust Contribution Evaluation in Federated Learning
University of La Laguna（拉拉加大学）；HUN-REN Hungarian Research Network（HUN-REN匈牙利研究网络）；Chalmers University（查尔姆斯大学）；Budapest University of Technology and Economics（布达佩斯技术经济大学）
Duke University（杜克大学）；Beihang University（北京航空航天大学）
The success of deep learning partially benefits from the availability of various large-scale datasets. These datasets are often crowdsourced from individual users and contain private information like gender, age, etc. The emerging privacy concerns from users on data sharing hinder the generation or use of crowdsourcing datasets and lead to hunger of training data for new deep learning applications. One na\"ıve solution is to pre-process the raw data to extract features at the user-side, and then only the extracted features will be sent to the data collector. Unfortunately, attackers can still …
Algorithms for oblivious random access machine (ORAM) simulation allow a client, Alice, to obfuscate a pattern of data accesses with a server, Bob, who is maintaining Alice's outsourced data while trying to learn information about her data. We present a novel ORAM scheme that improves the asymptotic I/O overhead of previous schemes for a wide range of size parameters for client-side private memory and message blocks, from logarithmic to polynomial. Our method achieves statistical security for hiding Alice's access pattern and, with high probability, achieves an I/O overhead that ranges from $O…
Recent advances in computing have allowed for the possibility to collect large amounts of data on personal activities and private living spaces. To address the privacy concerns of users in this environment, we propose a novel framework called PR-GAN that offers privacy-preserving mechanism using generative adversarial networks. Given a target application, PR-GAN automatically modifies the data to hide sensitive attributes -- which may be hidden and can be inferred by machine learning algorithms -- while preserving the data utility in the target application. Unlike prior works, the public's pos…
When sharing sensitive relational databases with other parties, a database owner aims to (i) have privacy guarantees for the database entries, (ii) have liability guarantees (via fingerprinting) in case of unauthorized sharing of its database by the recipients, and (iii) provide a high quality (utility) database to the recipients. We observe that sharing a relational database with privacy and liability guarantees are orthogonal objectives. The former can be achieved by injecting noise into the database to prevent inference of the original data values, whereas, the latter can be achieved by hid…
This paper aims at answering the following two questions in privacy-preserving data analysis and publishing: What formal privacy guarantee (if any) does $k$-anonymization provide? How to benefit from the adversary's uncertainty about the data? We have found that random sampling provides a connection that helps answer these two questions, as sampling can create uncertainty. The main result of the paper is that $k$-anonymization, when done "safely", and when preceded with a random sampling step, satisfies $(ε,δ)$-differential privacy with reasonable parameters. This result illustrates that "hidi…
Remotely generated data by Intent of Things (IoT) has recently had a lot of attention for their huge benefits such as efficient monitoring and risk reduction. The transmitted streams usually consist of periodical streams (e.g. activities) and highly private information (e.g. IDs). Despite the obvious benefits, the concerns are the secrecy and the originality of the transferred data. Surprisingly, although these concerns have been well studied for static data, they have received only limited attention for streaming data. Therefore, this paper introduces a new steganographic mechanism that provi…
Recent works in social network stream analysis show that a user's online persona attributes (e.g., gender, ethnicity, political interest, location, etc.) can be accurately inferred from the topics the user writes about or engages with. Attribute and preference inferences have been widely used to serve personalized recommendations, directed ads, and to enhance the user experience in social networks. However, revealing a user's sensitive attributes could represent a privacy threat to some individuals. Microtargeting (e.g.,Cambridge Analytica scandal), surveillance, and discriminating ads are exa…
Federated Learning (FL) is a Machine Learning (ML) technique that aims to reduce the threats to user data privacy. Training is done using the raw data on the users' device, called clients, and only the training results, called gradients, are sent to the server to be aggregated and generate an updated model. However, we cannot assume that the server can be trusted with private information, such as metadata related to the owner or source of the data. So, hiding the client information from the server helps reduce privacy-related attacks. Therefore, the privacy of the client's identity, along with…
While location data is extremely valuable for various applications, disclosing it prompts serious threats to individuals' privacy. To limit such concerns, organizations often provide analysts with aggregate time-series that indicate, e.g., how many people are in a location at a time interval, rather than raw individual traces. In this paper, we perform a measurement study to understand Membership Inference Attacks (MIAs) on aggregate location time-series, where an adversary tries to infer whether a specific user contributed to the aggregates. We find that the volume of contributed data, as wel…
Verifiable Fine-Tuning for LLMs: Zero-Knowledge Training Proofs Bound to Data Provenance and Policy
1 Department of Computer Engineering, Istanbul Technical University (ITU), 34469 Istanbul, Turkey Email；2 Department of Computer Science, University of Malta, MSD 2080 Msida, Malta Email；3 Faculty of Electrical \& Computer Engineering, University of Prishtina ``Hasan Prishtina'', 10000 Prishtina, Kosovo Email；4 School of IT \& Engineering, ADA University, AZ1008 Baku, Azerbaijan Email；5 Faculty of Mathematics, Natural Sciences；Information Technologies (FAMNIT), University of Primorska, 6000 Koper, Slovenia Email；6 Faculty of Computer Science \& Engineering (FINKI), Ss.\ Cyril；Methodius University in Skopje, 1000 Skopje, North Macedonia Email
