Privacy is under threat from artificial intelligence revolution fueled by unprecedented abundance of data. Differential privacy, an established candidate for privacy protection, is susceptible to adversarial attacks, acts conservatively, and leads to miss-implementations because of lacking systematic methods for setting its parameters (known as the privacy budget). An alternative is information-theoretic privacy using entropy with the drawback of requiring prior distribution of the private data. Here, by using the Fisher information, information-theoretic privacy framework is extended to avoid unnecessary assumptions on the private data. The optimal privacy-preserving additive noise, extracted by minimizing the Fisher information, must follow the time-independent Schrodinger's equation. A fundamental trade-off between privacy and utility is also proved, reminiscent of the Heisenberg uncertainty principle.
Biometric recognition is a highly adopted technology to support different kinds of applications, ranging from security and access control applications to low enforcement applications. However, such systems raise serious privacy and data protection concerns. Misuse of data, compromising the privacy of individuals and/or authorized processing of data may be irreversible and could have severe consequences on the individual's rights to privacy and data protection. This is partly due to the lack of methods and guidance for the integration of data protection and privacy by design in the system devel…
Decentralized systems are a subset of distributed systems where multiple authorities control different components and no authority is fully trusted by all. This implies that any component in a decentralized system is potentially adversarial. We revise fifteen years of research on decentralization and privacy, and provide an overview of key systems, as well as key insights for designers of future systems. We show that decentralized designs can enhance privacy, integrity, and availability but also require careful trade-offs in terms of system complexity, properties provided, and degree of decent…
Since its introduction in 2006, differential privacy has emerged as a predominant statistical tool for quantifying data privacy in academic works. Yet despite the plethora of research and open-source utilities that have accompanied its rise, with limited exceptions, differential privacy has failed to achieve widespread adoption in the enterprise domain. Our study aims to shed light on the fundamental causes underlying this academic-industrial utilization gap through detailed interviews of 24 privacy practitioners across 9 major companies. We analyze the results of our survey to provide key fin…
Criteo AI Lab（Criteo AI实验室）；Meta Platforms, Inc.（Meta平台公司）；Criteo AI Lab and Hui Yang Affiliation: Meta Platforms, Inc.（Criteo AI实验室）
Designing data sharing mechanisms providing performance and strong privacy guarantees is a hot topic for the Online Advertising industry. Namely, a prominent proposal discussed under the Improving Web Advertising Business Group at W3C only allows sharing advertising signals through aggregated, differentially private reports of past displays. To study this proposal extensively, an open Privacy-Preserving Machine Learning Challenge took place at AdKDD'21, a premier workshop on Advertising Science with data provided by advertising company Criteo. In this paper, we describe the challenge tasks, th…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Carnegie Mellon University（卡内基梅隆大学）；Microsoft Research（微软研究院）
Private regression has received attention from both database and security communities. Recent work by Fredrikson et al. (USENIX Security 2014) analyzed the functional mechanism (Zhang et al. VLDB 2012) for training linear regression models over medical data. Unfortunately, they found that model accuracy is already unacceptable with differential privacy when $\varepsilon = 5$. We address this issue, presenting an explicit connection between differential privacy and stable learning theory through which a substantially better privacy/utility tradeoff can be obtained. Perhaps more importantly, our…
We summarize the experience of participating in two differential privacy competitions organized by the National Institute of Standards and Technology (NIST). In this paper, we document our experiences in the competition, the approaches we have used, the lessons we have learned, and our call to the research community to further bridge the gap between theory and practice in DP research.
Sixth-generation (6G) mobile networks will have to cope with diverse threats on a space-air-ground integrated network environment, novel technologies, and an accessible user information explosion. However, for now, security and privacy issues for 6G remain largely in concept. This survey provides a systematic overview of security and privacy issues based on prospective technologies for 6G in the physical, connection, and service layers, as well as through lessons learned from the failures of existing security architectures and state-of-the-art defenses. Two key lessons learned are as follows. …
Preserving privacy of users is a key requirement of web-scale analytics and reporting applications, and has witnessed a renewed focus in light of recent data breaches and new regulations such as GDPR. We focus on the problem of computing robust, reliable analytics in a privacy-preserving manner, while satisfying product requirements. We present PriPeARL, a framework for privacy-preserving analytics and reporting, inspired by differential privacy. We describe the overall design and architecture, and the key modeling components, focusing on the unique challenges associated with privacy, coverage…
There has been vigorous debate on how different countries responded to the COVID-19 pandemic. To secure public safety, South Korea actively used personal information at the risk of personal privacy whereas France encouraged voluntary cooperation at the risk of public safety. In this article, after a brief comparison of contextual differences with France, we focus on South Korea's approaches to epidemiological investigations. To evaluate the issues pertaining to personal privacy and public health, we examine the usage patterns of original data, de-identification data, and encrypted data. Our sp…
Providing a provable privacy guarantees while maintaining the utility of data is a challenging task in many real-world applications. Recently, a new framework called One-Sided Differential Privacy (OSDP) was introduced that extends existing differential privacy approaches. OSDP increases the utility of the data by taking advantage of the fact that not all records are sensitive. However, the previous work assumed that all records are statistically independent from each other. Motivated by occupancy data in building management systems, this paper extends the existing one-sided differential priva…
Location-based services (LBSs) have become widely popular. Despite their utility, these services raise concerns for privacy since they require sharing location information with untrusted third parties. In this work, we study privacy-utility trade-off in location sharing mechanisms. Existing approaches are mainly focused on privacy of sharing a single location or myopic location trace privacy; neither of them taking into account the temporal correlations between the past and current locations. Although these methods preserve the privacy for the current time, they may leak significant amount of …
Taking into account of both the huge computing power of intruders and untrusted cloud servers, we develop an enhanced secure pseudonym scheme to protect the privacy of mobile cloud data. To face the huge computing power challenge, we develop an unconditionally secure lightweight network coding pseudonym scheme. For the privacy issue of untrusted cloud server, we further design a two tier network coding to decouple the stored mobile cloud data from the owner pseudonyms. Therefore, our proposed network coding based pseudonym scheme can simultaneously defend against attackers from both outside an…
Privacy by design will become a legal obligation in the European Community if the Data Protection Regulation eventually gets adopted. However, taking into account privacy requirements in the design of a system is a challenging task. We propose an approach based on the specification of privacy architectures and focus on a key aspect of privacy, data minimisation, and its tension with integrity requirements. We illustrate our formal framework through a smart metering case study.
Information-Centric Networking (ICN) is a new networking paradigm, which replaces the widely used host-centric networking paradigm in communication networks (e.g., Internet, mobile ad hoc networks) with an information-centric paradigm, which prioritizes the delivery of named content, oblivious of the contents origin. Content and client security are more intrinsic in the ICN paradigm versus the current host centric paradigm where they have been instrumented as an after thought. By design, the ICN paradigm inherently supports several security and privacy features, such as provenance and identity…
The use of software applications is inevitable as they provide different services to users. The software applications collect, store users' data, and sometimes share with the third party, even without the user consent. One can argue that software developers do not implement privacy into the software applications they develop or take GDPR (General Data Protection Law) law into account. Failing to do this, may lead to software applications that open up privacy breaches (e.g. data breach). The GDPR law provides a set of guidelines for developers and organizations on how to protect user data when …
We estimate vehicular traffic states from multimodal data collected by single-loop detectors while preserving the privacy of the individual vehicles contributing to the data. To this end, we propose a novel hybrid differential privacy (DP) approach that utilizes minimal randomization to preserve privacy by taking advantage of the relevant traffic state dynamics and the concept of DP sensitivity. Through theoretical analysis and experiments with real-world data, we show that the proposed approach significantly outperforms the related baseline non-private and private approaches in terms of accur…
In recent years, many proposals have arisen from research on privacy in smart metering. In one of the considered approaches, referred to as anonymization, smart meters transmit fine-grained electricity consumption values in such a way that the energy supplier can not exactly determine procedence. This paper measures the real privacy provided by such approach by taking into account that at the end of a billing period the energy supplier collects the overall electricity consumption of each meter for billing purposes. An entropy-based measure is proposed for quantifying privacy and determine the …
Technische Universität Berlin（柏林工业大学）；anacision GmbH（anacision有限公司）
An exploratory data analysis is an essential step for every data analyst to gain insights, evaluate data quality and (if required) select a machine learning model for further processing. While privacy-preserving machine learning is on the rise, more often than not this initial analysis is not counted towards the privacy budget. In this paper, we quantify the privacy loss for basic statistical functions and highlight the importance of taking it into account when calculating the privacy-loss budget of a machine learning approach.
With smartphone technologies enhanced way of interacting with the world around us, it has also been paving the way for easier access to our private and personal information. This has been amplified by the existence of numerous embedded sensors utilized by millions of apps to users. While mobile apps have positively transformed many aspects of our lives with new functionalities, many of these applications are taking advantage of vast amounts of data, privacy apps, a form of Privacy Enhancing Technology can be an effective privacy management tool for smartphones. To protect against vulnerabiliti…
Decentralized learning is an efficient emerging paradigm for boosting the computing capability of multiple bounded computing agents. In the big data era, performing inference within the distributed and federated learning (DL and FL) frameworks, the central server needs to process a large amount of data while relying on various agents to perform multiple distributed training tasks. Considering the decentralized computing topology, privacy has become a first-class concern. Moreover, assuming limited information processing capability for the agents calls for a sophisticated \textit{privacy-preser…
High-dimensional crowdsourced data collected from a large number of users produces rich knowledge for our society. However, it also brings unprecedented privacy threats to participants. Local privacy, a variant of differential privacy, is proposed as a means to eliminate the privacy concern. Unfortunately, achieving local privacy on high-dimensional crowdsourced data raises great challenges on both efficiency and effectiveness. Here, based on EM and Lasso regression, we propose efficient multi-dimensional joint distribution estimation algorithms with local privacy. Then, we develop a Locally p…
The proliferation of real-time applications has motivated extensive research on analyzing and optimizing data freshness in the context of \textit{age of information}. However, classical frameworks of privacy (e.g., differential privacy (DP)) have overlooked the impact of data freshness on privacy guarantees, which may lead to unnecessary accuracy loss when trying to achieve meaningful privacy guarantees in time-varying databases. In this work, we introduce \textit{age-dependent DP}, taking into account the underlying stochastic nature of a time-varying database. In this new framework, we estab…
A Smart Home provides integrating and electronic information services to help residential users manage their energy usage and bill cost but also exposes users to significant privacy risks due to fine-grained information collected by smart meters. Taking account of users' privacy concerns, this paper focuses on cost-effective runtime scheduling designed for schedulable and non-schedulable appliances. To alleviate the influence of operation uncertainties introduced by non-schedulable appliances, we formulate the problem by minimizing the expected sum of electricity cost under the worst privacy s…
Purdue University（普渡大学）；Eindhoven University of Technology（埃因霍温理工大学）
When collecting information, local differential privacy (LDP) relieves the concern of privacy leakage from users' perspective, as user's private information is randomized before sent to the aggregator. We study the problem of recovering the distribution over a numerical domain while satisfying LDP. While one can discretize a numerical domain and then apply the protocols developed for categorical domains, we show that taking advantage of the numerical nature of the domain results in better trade-off of privacy and utility. We introduce a new reporting mechanism, called the square wave SW mechan…
The advent of smartphones in recent years has changed the wireless landscape. Smartphones have become a platform for online user interface to cloud databases. Cloud databases may provide a large set of user-private and sensitive data (i.e., objects), while smartphone users (i.e., subjects) provide location-sensitive information. Secure and private services in wireless accessing to cloud databases have been discussed actively for the past recent years. However, the previous techniques are unsatisfactory for dynamism of moving subjects' wireless accesses. In this paper, we propose a novel techni…
In the field of machine learning, many problems can be formulated as the minimax problem, including reinforcement learning, generative adversarial networks, to just name a few. So the minimax problem has attracted a huge amount of attentions from researchers in recent decades. However, there is relatively little work on studying the privacy of the general minimax paradigm. In this paper, we focus on the privacy of the general minimax setting, combining differential privacy together with minimax optimization paradigm. Besides, via algorithmic stability theory, we theoretically analyze the high …
Mobile edge computing (MEC) has been envisioned as a promising paradigm to handle the massive volume of data generated from ubiquitous mobile devices for enabling intelligent services with the help of artificial intelligence (AI). Traditionally, AI techniques often require centralized data collection and training in a single entity, e.g., an MEC server, which is now becoming a weak point due to data privacy concerns and high data communication overheads. In this context, federated learning (FL) has been proposed to provide collaborative data training solutions, by coordinating multiple mobile …
UCLA Electrical and Computer Engineering Department（加州大学洛杉矶分校电气与计算机工程系）；Amazon Web Services（亚马逊云科技）
Data privacy is an important concern in learning, when datasets contain sensitive information about individuals. This paper considers consensus-based distributed optimization under data privacy constraints. Consensus-based optimization consists of a set of computational nodes arranged in a graph, each having a local objective that depends on their local data, where in every step nodes take a linear combination of their neighbors' messages, as well as taking a new gradient step. Since the algorithm requires exchanging messages that depend on local data, private information gets leaked at every …
Massachusetts Institute of Technology（麻省理工学院）；MIT Energy Initiative（麻省理工学院能源倡议）
While power systems research relies on the availability of real-world network datasets, data owners (e.g., system operators) are hesitant to share data due to security and privacy risks. To control these risks, we develop privacy-preserving algorithms for the synthetic generation of optimization and machine learning datasets. Taking a real-world dataset as input, the algorithms output its noisy, synthetic version, which preserves the accuracy of the real data on a specific downstream model or even a large population of those. We control the privacy loss using Laplace and Exponential mechanisms…
Data-driven systems are gathering increasing amounts of data from users, and sensitive user data requires privacy protections. In some cases, the data gathered is non-numerical or symbolic, and conventional approaches to privacy, e.g., adding noise, do not apply, though such systems still require privacy protections. Accordingly, we present a novel differential privacy framework for protecting trajectories generated by symbolic systems. These trajectories can be represented as words or strings over a finite alphabet. We develop new differential privacy mechanisms that approximate a sensitive w…
When it comes to location-based services (LBS), user privacy protection can be in conflict with security of both users and trips. While LBS providers could adopt privacy preservation mechanisms to obfuscate customer data, the accuracy of vehicle location data and trajectories is crucial for detecting anomalies, especially when machine learning methods are adopted by LBS. This paper aims to tackle this dilemma by evaluating the tradeoff between location privacy and security in LBS. In particular, we investigate the impact of applying location data privacy-preservation techniques on the performa…
The use of connected surgical robotics to automate medical procedures presents new privacy challenges. We argue that conventional patient consent protocols no longer work. Indeed robots that replace human surgeons take on an extraordinary level of responsibility. Surgeons undergo years of training and peer review in a strongly regulated environment, and derive trust via a patient's faith in the hospital system. Robots on the other hand derive trust differently, via the integrity of the software that governs their operation. From a privacy perspective, there are two fundamental shifts. First, t…
Andalusian Research Institute in Data Science and Computational Intelligence（安达卢西亚数据科学与计算智能研究院）；University of Granada（格拉纳达大学）
Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real…
Stanford University（斯坦福大学）；Microsoft Research（微软研究院）；U.S. Bureau of Economic Analysis（美国经济分析局）
Estimating heterogeneous treatment effects in domains such as healthcare or social science often involves sensitive data where protecting privacy is important. We introduce a general meta-algorithm for estimating conditional average treatment effects (CATE) with differential privacy (DP) guarantees. Our meta-algorithm can work with simple, single-stage CATE estimators such as S-learner and more complex multi-stage estimators such as DR and R-learner. We perform a tight privacy analysis by taking advantage of sample splitting in our meta-algorithm and the parallel composition property of differ…
In this chapter, we will explore the cloud-outsourced privacy-preserving computation of a controller on encrypted measurements from a (possibly distributed) system, taking into account the challenges introduced by the dynamical nature of the data. The privacy notion used in this work is that of cryptographic multi-party privacy, i.e., the computation of a functionality should not reveal anything more than what can be inferred only from the inputs and outputs of the functionality. The main theoretical concept used towards this goal is Homomorphic Encryption, which allows the evaluation of sums …
We consider non-parametric density estimation in the framework of local approximate differential privacy. In contrast to centralized privacy scenarios with a trusted curator, in the local setup anonymization must be guaranteed already on the individual data owners' side and therefore must precede any data mining tasks. Thus, the published anonymized data should be compatible with as many statistical procedures as possible. We suggest adding Laplace noise and Gaussian processes (both appropriately scaled) to kernel density estimators to obtain approximate differential private versions of the la…
University of California, Berkeley（加州大学伯克利分校）；Virginia Tech（弗吉尼亚理工大学）
In this paper, we propose the discrete Gaussian based differentially private federated learning (D2P-Fed), a unified scheme to achieve both differential privacy (DP) and communication efficiency in federated learning (FL). In particular, compared with the only prior work taking care of both aspects, D2P-Fed provides stronger privacy guarantee, better composability and smaller communication cost. The key idea is to apply the discrete Gaussian noise to the private data transmission. We provide complete analysis of the privacy guarantee, communication cost and convergence rate of D2P-Fed. We eval…
