In this paper, we study the fundamental limits of obfuscation in terms of privacy-distortion tradeoffs for linear Gaussian dynamical systems via an information-theoretic approach. Particularly, we obtain analytical formulas that capture the fundamental privacy-distortion tradeoffs when privacy masks are to be added to the outputs of the dynamical systems, while indicating explicitly how to design the privacy masks in an optimal way: The privacy masks should be colored Gaussian with power spectra shaped specifically based upon the system and noise properties.
In this paper, we first introduce the notion of channel leakage as the minimum mutual information between the channel input and channel output. As its name indicates, channel leakage quantifies the minimum information leakage to the malicious receiver. In a broad sense, it can be viewed as a dual concept of channel capacity, which characterizes the maximum information transmission to the targeted receiver. We obtain explicit formulas of channel leakage for the white Gaussian case, the colored Gaussian case, and the fading case. We then utilize this notion to investigate the fundamental limitat…
UC San Diego（加州大学圣地亚哥分校）；Qualcomm AI Research（高通人工智能研究院）
Splitting network computations between the edge device and a server enables low edge-compute inference of neural networks but might expose sensitive information about the test query to the server. To address this problem, existing techniques train the model to minimize information leakage for a given set of sensitive attributes. In practice, however, the test queries might contain attributes that are not foreseen during training. We propose instead an unsupervised obfuscation method to discard the information irrelevant to the main task. We formulate the problem via an information theoretical …
The re-identification or de-anonymization of users from anonymized data through matching with publicly-available correlated user data has raised privacy concerns, leading to the complementary measure of obfuscation in addition to anonymization. Recent research provides a fundamental understanding of the conditions under which privacy attacks are successful, either in the presence of obfuscation or synchronization errors stemming from the sampling of time-indexed databases. This paper presents a unified framework considering both obfuscation and synchronization errors and investigates the match…
We discuss why MMSE estimation arises in lattice-based schemes for approaching the capacity of linear Gaussian channels, and comment on its properties.
This paper focuses on three contributions. First, a connection between the result, proposed by Stoica and Babu, and the recent information theoretic results, the worst additive noise lemma and the isoperimetric inequality for entropies, is illustrated. Second, information theoretic and estimation theoretic justifications for the fact that the Gaussian assumption leads to the largest Cramér-Rao lower bound (CRLB) is presented. Third, a slight extension of this result to the more general framework of correlated observations is shown.
The behavior of rational and selfish players (receivers) over a multiple-input multiple-output Gaussian broadcast channel is investigated using the framework of noncooperative game theory. In contrast to the game-theoretic model of the Gaussian multiple access channel where the set of feasible actions for each player is independent of other players' actions, the strategies of the players in the broadcast channel are mutually coupled, usually by a sum power or joint covariance constraint, and hence cannot be treated using traditional Nash equilibrium solution concepts. To characterize the strat…
The theoretical foundations of data hiding have been revealed by formulating the problem as message communication over a noisy channel. We revisit the problem in light of a more general characterization of the watermark channel and of weighted distortion measures. Considering spread spectrum based information hiding, we release the usual assumption of an i.i.d. cover signal. The game-theoretic resolution of the problem reveals a generalized characterization of optimum attacks. The paper then derives closed-form expressions for the different parameters exhibiting a practical embedding and extra…
Louisiana State University（路易斯安那州立大学）；Iowa State University（爱荷华州立大学）；University of North Carolina at Greensboro（北卡罗来纳大学格林斯伯勒分校）
In latent Gaussian trees the pairwise correlation signs between the variables are intrinsically unrecoverable. Such information is vital since it completely determines the direction in which two variables are associated. In this work, we resort to information theoretical approaches to achieve two fundamental goals: First, we quantify the amount of information loss due to unrecoverable sign information. Second, we show the importance of such information in determining the maximum achievable rate region, in which the observed output vector can be synthesized, given its probability density functi…
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified approach to functional inequalities such as the variational formula of Rényi entropy, hypercontractivity and its reverse, strong data processing inequalities, and transportation-cost inequalities, whose utility in the proofs of various coding theorems has gained growing popularity recently. We show that our information-theoretic setting is convenient for pro…
In this paper, the compression of an independent and identically distributed Gaussian source sequence is studied in an unsecure network. Within a game theoretic setting for a three-party noiseless communication network (sender Alice, legitimate receiver Bob, and eavesdropper Eve), the problem of how to efficiently compress a Gaussian source with limited secret key in order to guarantee that Bob can reconstruct with high fidelity while preventing Eve from estimating an accurate reconstruction is investigated. It is assumed that Alice and Bob share a secret key with limited rate. Three scenarios…
What is the optimal number of independent observations from which a sparse Gaussian Graphical Model can be correctly recovered? Information-theoretic arguments provide a lower bound on the minimum number of samples necessary to perfectly identify the support of any multivariate normal distribution as a function of model parameters. For a model defined on a sparse graph with $p$ nodes, a maximum degree $d$ and minimum normalized edge strength $κ$, this necessary number of samples scales at least as $d \log p/κ^2$. The sample complexity requirements of existing methods for perfect graph reconstr…
The synergetic gains of spectrum sharing and millimeter wave communication networks have recently attracted attention, owing to the interference canceling benefits of highly-directional beamforming in such systems. In principle, fine-tuned coordinated scheduling and beamforming can drastically reduce cross-operator interference. However, this goes at the expense of the exchange of global channel state information, which is not realistic in particular when considering inter-operator coordination. Indeed, such an exchange of information is expensive in terms of backhaul infrastructure, and besid…
We consider the transmission of a common message from a transmitter to three receivers over a broadcast channel, referred to as a multicast channel in this case. All the receivers are allowed to cooperate with each other over full-duplex bi-directional non-orthogonal cooperation links. We investigate the information-theoretic upper and lower bounds on the transmission rate. In particular, we propose a three-receiver fully interactive cooperation scheme (3FC) based on superpositions of CF and DF at the receivers. In the 3FC scheme, the receivers interactively perform compress-forward (CF) simul…
This paper studies the capacity of an $n$-dimensional vector Gaussian noise channel subject to the constraint that an input must lie in the ball of radius $R$ centered at the origin. It is known that in this setting the optimizing input distribution is supported on a finite number of concentric spheres. However, the number, the positions and the probabilities of the spheres are generally unknown. This paper characterizes necessary and sufficient conditions on the constraint $R$ such that the input distribution supported on a single sphere is optimal. The maximum $\bar{R}_n$, such that using on…
Rate-distortion formulation is the information-theoretic approach to the study of signal encoding systems. Since a more general approach to model the nonstationarity exhibited by real-world signals is to use appropriately fitted time varying autoregressive (TVAR) models, we have investigated the rate-distortion function for the class of time varying nonstationary signals. In this study, we present formulations of the rate-distortion function for the Gaussian TVAR processes. The rate-distortion function can serve as an information-theoretic bound on the performance achievable by source encoding…
We produce a series of results extending information-theoretical inequalities (discussed by Dembo--Cover--Thomas in 1989-1991) to a weighted version of entropy. The resulting inequalities involve the Gaussian weighted entropy; they imply a number of new relations for determinants of positive-definite matrices.
This work studies anomaly detection under differential privacy (DP) with Gaussian perturbation using both statistical and information-theoretic tools. In our setting, the adversary aims to modify the content of a statistical dataset by inserting additional data without being detected by using the DP guarantee to her own benefit. To this end, we characterize information-theoretic and statistical thresholds for the first and second-order statistics of the adversary's attack, which balances the privacy budget and the impact of the attack in order to remain undetected. Additionally, we introduce a…
We generalize a result by Carlen and Cordero-Erausquin on the equivalence between the Brascamp-Lieb inequality and the subadditivity of relative entropy by allowing for random transformations (a broadcast channel). This leads to a unified perspective on several functional inequalities that have been gaining popularity in the context of proving impossibility results. We demonstrate that the information theoretic dual of the Brascamp-Lieb inequality is a convenient setting for proving properties such as data processing, tensorization, convexity and Gaussian optimality. Consequences of the latter…
In a two-user channel, completion time refers to the number of channel uses spent by each user to transmit a bit pool with some given size. In this paper, the information-theoretic formulation of completion time is based on the concept of constrained rates, where users are allowed to employ different numbers of channel uses for transmission as opposed to the equal channel use of the standard information-theoretic formulation. Analogous to the capacity region, the completion time region characterizes all possible trade-offs among users' completion times. For a multi-access channel, it is shown …
A renowned information-theoretic formula by Shannon expresses the mutual information rate of a white Gaussian channel with a stationary Gaussian input as an integral of a simple function of the power spectral density of the channel input. We give in this paper a rigorous yet elementary proof of this classical formula. As opposed to all the conventional approaches, which either rely on heavy mathematical machineries or have to resort to some "external" results, our proof, which hinges on a recently proven sampling theorem, is elementary and self-contained, only using some well-known facts from …
We determine the information-theoretic cutoff value on separation of cluster centers for exact recovery of cluster labels in a $K$-component Gaussian mixture model with equal cluster sizes. Moreover, we show that a semidefinite programming (SDP) relaxation of the $K$-means clustering method achieves such sharp threshold for exact recovery without assuming the symmetry of cluster centers.
