Deep learning models leak significant amounts of information about their training datasets. Previous work has investigated training models with differential privacy (DP) guarantees through adding DP noise to the gradients. However, such solutions (specifically, DPSGD), result in large degradations in the accuracy of the trained models. In this paper, we aim at training deep learning models with DP guarantees while preserving model accuracy much better than previous works. Our key technique is to encode gradients to map them to a smaller vector space, therefore enabling us to obtain DP guarantees for different noise distributions. This allows us to investigate and choose noise distributions that best preserve model accuracy for a target privacy budget. We also take advantage of the post-processing property of differential privacy by introducing the idea of denoising, which further improves the utility of the trained models without degrading their DP guarantees. We show that our mechanism outperforms the state-of-the-art DPSGD; for instance, for the same model accuracy of $96.1\%$ on MNIST, our technique results in a privacy bound of $ε=3.2$ compared to $ε=6$ of DPSGD, which is a significant improvement.
Amazon Research（亚马逊研究院）；Amazon Web Services（亚马逊云科技）
The Gaussian mechanism is an essential building block used in multitude of differentially private data analysis algorithms. In this paper we revisit the Gaussian mechanism and show that the original analysis has several important limitations. Our analysis reveals that the variance formula for the original mechanism is far from tight in the high privacy regime ($\varepsilon \to 0$) and it cannot be extended to the low privacy regime ($\varepsilon \to \infty$). We address these limitations by developing an optimal Gaussian mechanism whose variance is calibrated directly using the Gaussian cumula…
University of Florida（佛罗里达大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
We present a framework to learn privacy-preserving encodings of images that inhibit inference of chosen private attributes, while allowing recovery of other desirable information. Rather than simply inhibiting a given fixed pre-trained estimator, our goal is that an estimator be unable to learn to accurately predict the private attributes even with knowledge of the encoding function. We use a natural adversarial optimization-based formulation for this---training the encoding function against a classifier for the private attribute, with both modeled as deep neural networks. The key contribution…
Over the last decade, signal processing on graphs has become a very active area of research. Specifically, the number of applications, for instance in statistical or deep learning, using frames built from graphs, such as wavelets on graphs, has increased significantly. We consider in particular the case of signal denoising on graphs via a data-driven wavelet tight frame methodology. This adaptive approach is based on a threshold calibrated using Stein's unbiased risk estimate adapted to a tight-frame representation. We make it scalable to large graphs using Chebyshev-Jackson polynomial approxi…
Graph Neural Networks (GNNs) have demonstrated superior performance in learning node representations for various graph inference tasks. However, learning over graph data can raise privacy concerns when nodes represent people or human-related variables that involve sensitive or personal information. While numerous techniques have been proposed for privacy-preserving deep learning over non-relational data, there is less work addressing the privacy issues pertained to applying deep learning algorithms on graphs. In this paper, we study the problem of node data privacy, where graph nodes have pote…
Two major challenges in distributed learning and estimation are 1) preserving the privacy of the local samples; and 2) communicating them efficiently to a central server, while achieving high accuracy for the end-to-end task. While there has been significant interest in addressing each of these challenges separately in the recent literature, treatments that simultaneously address both challenges are still largely missing. In this paper, we develop novel encoding and decoding mechanisms that simultaneously achieve optimal privacy and communication efficiency in various canonical settings. In pa…
University of Kalyani（卡利亚尼大学）；National Institute of Science, Education and Research（国家科学教育与研究所）；Homi Bhaba National Institute（霍米·巴巴国家研究所）
Balancing utility and differential privacy by shuffling or \textit{BUDS} is an approach towards crowd-sourced, statistical databases, with strong privacy and utility balance using differential privacy theory. Here, a novel algorithm is proposed using one-hot encoding and iterative shuffling with the loss estimation and risk minimization techniques, to balance both the utility and privacy. In this work, after collecting one-hot encoded data from different sources and clients, a step of novel attribute shuffling technique using iterative shuffling (based on the query asked by the analyst) and lo…
Machine learning has been applied to almost all fields of computer science over the past decades. The introduction of GANs allowed for new possibilities in fields of medical research and text prediction. However, these new fields work with ever more privacy-sensitive data. In order to maintain user privacy, a combination of federated learning, differential privacy and GANs can be used to work with private data without giving away a users' privacy. Recently, two implementations of such combinations have been published: DP-Fed-Avg GAN and GS-WGAN. This paper compares their performance and introd…
University College London（伦敦大学学院）；Alan Turing Institute（艾伦·图灵研究所）；Faculty（Faculty公司）
The collection of individuals' data has become commonplace in many industries. Local differential privacy (LDP) offers a rigorous approach to preserving privacy whereby the individual privatises their data locally, allowing only their perturbed datum to leave their possession. LDP thus provides a provable privacy guarantee to the individual against both adversaries and database administrators. Existing LDP mechanisms have successfully been applied to low-dimensional data, but in high dimensions the privacy-inducing noise largely destroys the utility of the data. In this work, our contributions…
Federated Diffusion Modeling with Differential Privacy for Tabular Data Synthesis
Deutsche Bundesbank（德国联邦银行）；University of St.Gallen（圣加尔大学）；Swiss Federal Audit Office（瑞士联邦审计局）；International Computer Science Institute (ICSI)（国际计算机科学研究所）
Seoul National University Hospital（首尔国立大学医院）；Chung-Ang University（中央大学）；Korea University（高丽大学）；Deakin University（迪肯大学）；University of Oulu（奥卢大学）；Yonsei University（延世大学）
User-generated data distributions are often imbalanced across devices and labels, hampering the performance of federated learning (FL). To remedy to this non-independent and identically distributed (non-IID) data problem, in this work we develop a privacy-preserving XOR based mixup data augmentation technique, coined XorMixup, and thereby propose a novel one-shot FL framework, termed XorMixFL. The core idea is to collect other devices' encoded data samples that are decoded only using each device's own data samples. The decoding provides synthetic-but-realistic samples until inducing an IID dat…
Federated learning has emerged as a popular paradigm for collaboratively training a model from data distributed among a set of clients. This learning setting presents, among others, two unique challenges: how to protect privacy of the clients' data during training, and how to ensure integrity of the trained model. We propose a two-pronged solution that aims to address both challenges under a single framework. First, we propose to create secure enclaves using a trusted execution environment (TEE) within the server. Each client can then encrypt their gradients and send them to verifiable enclave…
MPU: Towards Secure and Privacy-Preserving Knowledge Unlearning for Large Language Models
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Alibaba-NTU Global e-Sustainability CorpLab (ANGEL)（阿里云-南洋理工大学全球可持续发展科技实验室）；Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）
Compressive sensing is an impressive approach for fast MRI. It aims at reconstructing MR image using only a few under-sampled data in k-space, enhancing the efficiency of the data acquisition. In this study, we propose to learn priors based on undecimated wavelet transform and an iterative image reconstruction algorithm. At the stage of prior learning, transformed feature images obtained by undecimated wavelet transform are stacked as an input of denoising autoencoder network (DAE). The highly redundant and multi-scale input enables the correlation of feature images at different channels, whic…
Deep learning-based language models have achieved state-of-the-art results in a number of applications including sentiment analysis, topic labelling, intent classification and others. Obtaining text representations or embeddings using these models presents the possibility of encoding personally identifiable information learned from language and context cues that may present a risk to reputation or privacy. To ameliorate these issues, we propose Context-Aware Private Embeddings (CAPE), a novel approach which preserves privacy during training of embeddings. To maintain the privacy of text repres…
Montreal Institute for Learning Algorithms（蒙特利尔学习算法研究所）；University of Montreal（蒙特利尔大学）
Denoising autoencoders (DAE) are trained to reconstruct their clean inputs with noise injected at the input level, while variational autoencoders (VAE) are trained with noise injected in their stochastic hidden layer, with a regularizer that encourages this noise injection. In this paper, we show that injecting noise both in input and in the stochastic hidden layer can be advantageous and we propose a modified variational lower bound as an improved objective function in this setup. When input is corrupted, then the standard VAE lower bound involves marginalizing the encoder conditional distrib…
Differentially Private Non Parametric Copulas: Generating synthetic data with non parametric copulas under privacy guarantees
School of Applied Sciences and Engineering, Universidad EAFIT（应用科学与工程学院，埃法特大学）；Digital Health and Biomedical Technologies, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA)（数字健康与生物医学技术，维科姆科技基金会，巴斯克研究与技术联盟（BRTA））；eHealth Group, Biogipuzkoa Health Research Institute（eHealth组，比戈普西亚健康研究 institutes）
University of Tübingen（图宾根大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
Eye tracking is handled as one of the key technologies for applications that assess and evaluate human attention, behavior, and biometrics, especially using gaze, pupillary, and blink behaviors. One of the challenges with regard to the social acceptance of eye tracking technology is however the preserving of sensitive and personal information. To tackle this challenge, we employ a privacy-preserving framework based on randomized encoding to train a Support Vector Regression model using synthetic eye images privately to estimate the human gaze. During the computation, none of the parties learn …
A private machine learning algorithm hides as much as possible about its training data while still preserving accuracy. In this work, we study whether a non-private learning algorithm can be made private by relying on an instance-encoding mechanism that modifies the training inputs before feeding them to a normal learner. We formalize both the notion of instance encoding and its privacy by providing two attack models. We first prove impossibility results for achieving a (stronger) model. Next, we demonstrate practical attacks in the second (weaker) attack model on InstaHide, a recent proposal …
Due to patient privacy protection concerns, machine learning research in healthcare has been undeniably slower and limited than in other application domains. High-quality, realistic, synthetic electronic health records (EHRs) can be leveraged to accelerate methodological developments for research purposes while mitigating privacy concerns associated with data sharing. The current state-of-the-art model for synthetic EHR generation is generative adversarial networks, which are notoriously difficult to train and can suffer from mode collapse. Denoising Diffusion Probabilistic Models, a class of …
Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train or infer with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. Cloud systems are vulnerable to attackers that compromise the privacy of data and integrity of computations. Tackling such a challenge requires unifying theoretical privacy algorithms with hardware security capabilities. This paper presents DarKnight, a framework for large DNN training while protecting input privacy and computation integrity. DarKnight relie…
Indian Statistical Institute（印度统计研究所）；IBM Research（IBM研究院）
Amidst an increasing number of infected cases during the Covid-19 pandemic, it is essential to trace, as early as possible, the susceptible people who might have been infected by the disease due to their close proximity with people who were tested positive for the virus. This early contact tracing is likely to limit the rate of spread of the infection within a locality. In this paper, we investigate how effectively and efficiently can such a list of susceptible people be found given a list of infected persons and their locations. To address this problem from an information retrieval (search) p…
Cornell University（康奈尔大学）；National University of Singapore（新加坡国立大学）
Machine learning as a service (MLaaS), and algorithm marketplaces are on a rise. Data holders can easily train complex models on their data using third party provided learning codes. Training accurate ML models requires massive labeled data and advanced learning algorithms. The resulting models are considered as intellectual property of the model owners and their copyright should be protected. Also, MLaaS needs to be trusted not to embed secret information about the training data into the model, such that it could be later retrieved when the model is deployed. In this paper, we present \emph{m…
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿蒂努拉·A·马蒂诺斯生物医学成像中心）；Department of Radiology, Harvard Medical School（哈佛医学院放射学系）；Department of Biomedical Engineering, Tsinghua University（清华大学生物医学工程系）；Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology（麻省理工学院哈佛-麻省理工健康科学与技术部）
The noise in diffusion-weighted images (DWIs) decreases the accuracy and precision of diffusion tensor magnetic resonance imaging (DTI) derived microstructural parameters and leads to prolonged acquisition time for achieving improved signal-to-noise ratio (SNR). Deep learning-based image denoising using convolutional neural networks (CNNs) has superior performance but often requires additional high-SNR data for supervising the training of CNNs, which reduces the practical feasibility. We develop a self-supervised deep learning-based method entitled "SDnDTI" for denoising DTI data, which does n…
De Montfort University（德蒙福特大学）；AIT Austrian Institute of Technology（奥地利技术研究院）；Medical University Graz（格拉茨医科大学）；Université de Lorraine（洛林大学）
In the era of Internet of Things (IoT) technologies the potential for privacy invasion is becoming a major concern especially in regards to healthcare data and Ambient Assisted Living (AAL) environments. Systems that offer AAL technologies make extensive use of personal data in order to provide services that are context-aware and personalized. This makes privacy preservation a very important issue especially since the users are not always aware of the privacy risks they could face. A lot of progress has been made in the deep learning field, however, there has been lack of research on privacy p…
Cyclic Denoising Reveals Ultrastable Memories in Diffusion Models
Simons Center for Computational Physical Chemistry, New York University（纽约大学西蒙斯计算物理化学中心）；Center for Soft Matter Research, Department of Physics, New York University（纽约大学物理系软物质研究中心）；Center for Neural Science, New York University（纽约大学神经科学中心）；Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）
Smart sensors, devices and systems deployed in smart cities have brought improved physical protections to their citizens. Enhanced crime prevention, and fire and life safety protection are achieved through these technologies that perform motion detection, threat and actors profiling, and real-time alerts. However, an important requirement in these increasingly prevalent deployments is the preservation of privacy and enforcement of protection of personal identifiable information. Thus, strong encryption and anonymization techniques should be applied to the collected data. In this IEEE Big Data …
Unsupervised CP-UNet Framework for Denoising DAS Data with Decay Noise
School of Mechanical Engineering and Electronic Information, China University of Geosciences (Wuhan)（中国地质大学（武汉）机械工程与电子信息学院）；Shenzhen Research Institute of China University of Geosciences（中国地质大学深圳研究院）；Yazheng Technology Group Co., Ltd（雅正科技集团有限公司）；China Electric Power Planning & Engineering Institute（中国电力规划设计总院）；Liuzhou Wuling New Energy Automobile Co., Ltd（柳州五菱新能源汽车有限公司）；Yangtze optical fiber and cable Joint Stock Limited Company（长飞光纤光缆股份有限公司）；School of Economics and Management, China University of Geosciences (Wuhan)（中国地质大学（武汉）经济管理学院）
本文提出无监督 CP-UNet 框架，通过 Context Pyramid Module、CM、LN 与 Huber-loss 抑制 DAS 数据中的随机和不规则噪声，且无需标签，实验显示其降噪性能优于传统方法和现有 UL 框架。
