The problem of distributed binary hypothesis testing in the Gray-Wyner network with side information is studied in this paper. An observer has access to a discrete memoryless and stationary source and describes its observation to two detectors via one common and two private channels. The channels are considered error-free but rate-limited. Each detector also has access to its own discrete memoryless and stationary source, i.e., the side information. The goal is to perform two distinct binary hypothesis testings on the joint distribution of observations at detectors. Additionally, the observer aims to keep a correlated latent source private against the detectors. Equivocation is used as the measure of the privacy preserved for the latent source. An achievable inner bound is derived for the general case by introducing a non-asymptotic account of the output statistics of the random binning.
We show a reduction method to construct a code for the Gray-Wyner (GW) network from a given code for the Wyner-Ahlswede-Körner (WAK) network. By combining this reduction with a converse bound on the GW network, we derive a converse bound on the WAK network. The derived bound gives an alternative proof of the strong converse theorem for the WAK network.
The Gray-Wyner network subject to a fidelity criterion is studied. Upper and lower bounds for the trade-offs between the private sum-rate and the common rate are obtained for arbitrary sources subject to mean-squared error distortion. The bounds meet exactly, leading to the computation of the rate region, when the source is jointly Gaussian. They meet partially when the sources are modeled via an additive Gaussian "channel". The bounds are inspired from the Shannon bounds on the rate-distortion problem.
In this paper, the rate-distortion theory of the Gray-Wyner lossy source coding system is investigated. For the case of jointly Gaussian distributed sources, we establish an expression for the rate-distortion function under the constraint of quadratic distortion. Using the proposed rate-distortion function, any corner point on the rate-distortion region can be conveniently calculated. We take Wyner's common information as an example and provide a general and simple method to solve this problem. Through the analysis of the rate-distortion function, the rate on each layer and covariance matrix o…
We establish the rate region of an extended Gray-Wyner system for 2-DMS $(X,Y)$ with two additional decoders having complementary causal side information. This extension is interesting because in addition to the operationally significant extreme points of the Gray-Wyner rate region, which include Wyner's common information, G{á}cs-K{ö}rner common information and information bottleneck, the rate region for the extended system also includes the K{ö}rner graph entropy, the privacy funnel and excess functional information, as well as three new quantities of potential interest, as extreme points. T…
The coding problem over the Gray-Wyner network is studied from the second-order coding rates perspective. A tilted information density for this network is introduced in the spirit of Kostina-Verdú, and, under a certain regularity condition, the second-order region is characterized in terms of the variance of this tilted information density and the tangent vector of the first-order region. The second-order region is proved by the type method: the achievability part is proved by the type-covering argument, and the converse part is proved by a refinement of the perturbation approach that was used…
We consider the problem of source coding subject to a fidelity criterion for the Gray-Wyner network that connects a single source with two receivers via a common channel and two private channels. The pareto-optimal trade-offs between the sum-rate of the private channels and the rate of the common channel is completely characterized for jointly Gaussian sources subject to the mean-squared error criterion, leveraging convex duality and an argument involving the factorization of convex envelopes. Specifically, it is attained by selecting the auxiliary random variable to be jointly Gaussian with t…
The problem of preserving privacy when a multivariate source is required to be revealed partially to multiple users is modeled as a Gray-Wyner source coding problem with K correlated sources at the encoder and K decoders in which the kth decoder, k = 1, 2, ...,K, losslessly reconstructs the kth source via a common link and a private link. The privacy requirement of keeping each decoder oblivious of all sources other than the one intended for it is introduced via an equivocation constraint at each decoder such that the total equivocation summed over all decoders is E. The set of achievable rate…
In decentralized decision-making problems, communicating agents choose their actions based on locally available information and knowledge about decision rules or strategies of other agents. In this work, we consider a strategic communication game between an informed encoder and two decoders communicating via a Gray-Wyner network. All three agents are assumed to be rational and endowed with distinct objectives captured by non-aligned cost functions. The encoder selects and announces beforehand the compression scheme to be implemented. Then, it transmits three signals: a public signal, and a pri…
Explicit constructions of polar codes and polar lattices for both lossless and lossy Gray-Wyner problems are studied. Polar codes are employed to extract Wyner's common information of doubly symmetric binary source; polar lattices are then extended to extract that of a pair of Gaussian sources or multiple Gaussian sources. With regard to the discrete sources, the entire best-known region of the lossless Gray-Wyner problem are achieved by specifying the test channels to construct polar codes without time-sharing. As a result, we are able to give an interpretation that the Wyner's common informa…
Examined in this paper, is the Gray and Wyner achievable lossy rate region for a tuple of correlated multivariate Gaussian random variables (RVs) $X_1 : Ω\rightarrow {\mathbb R}^{p_1}$ and $X_2 : Ω\rightarrow {\mathbb R}^{p_2}$ with respect to square-error distortions at the two decoders. It is shown that among all joint distributions induced by a triple of RVs $(X_1,X_2, W)$, such that $W : Ω\rightarrow {\mathbb W} $ is the auxiliary RV taking continuous, countable, or finite values, the Gray and Wyner achievable rate region is characterized by jointly Gaussian RVs $(X_1,X_2, W)$ such that $W…
Rate-Distortion-Perception Tradeoff for the Gray-Wyner Problem
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology（南方科技大学自动化与智能制造学院）；School of Cyber Science and Technology, Beihang University（北京航空航天大学网络空间科学学院）
In this paper, we revisit the discrete lossy Gray-Wyner problem. In particular, we derive its optimal second-order coding rate region, its error exponent (reliability function) and its moderate deviations constant under mild conditions on the source. To obtain the second-order asymptotics, we extend some ideas from Watanabe's work (2015). In particular, we leverage the properties of an appropriate generalization of the conditional distortion-tilted information density, which was first introduced by Kostina and Verdú (2012). The converse part uses a perturbation argument by Gu and Effros (2009)…
We analyse the common information problem for the generalised Gray-Wyner problem. We aim to explore the problem and solution in relation to the non-orthogonality among the source decoders' components. We consider a simple networked control system consisting of 2 groups of users: (i) one sender or Observer named Alice; and (ii) a group of multiple receivers or Controllers, named Bobs. In order to tackle the possible risk arisen from the common information among Bobs, Alice provides a redundancy creating some virtual messages which are in the null of each specific Bob, but not for others. The af…
The two most prevalent notions of common information (CI) are due to Wyner and Gacs-Korner and both the notions can be stated as two different characteristic points in the lossless Gray-Wyner region. Although the information theoretic characterizations for these two CI quantities can be easily evaluated for random variables with infinite entropy (eg., continuous random variables), their operational significance is applicable only to the lossless framework. The primary objective of this paper is to generalize these two CI notions to the lossy Gray-Wyner network, which hence extends the theoreti…
In this work, we investigate two source coding models, a \emph{Helper} problem and a \emph{Gray-Wyner} problem, under equivocation constraints. Specifically, in the Helper problem, an encoder communicates with a legitimate receiver through noise-free rate-limited public and private links; and an external passive eavesdropper intercepts every information that is sent on the public link. We study two classes of this model: i) when a pair of arbitrarily correlated discrete memoryless sources is to be encoded such that one component has to be recovered lossily at the legitimate receiver while the …
In this work, we establish a full single-letter characterization of the rate-distortion region of an instance of the Gray-Wyner model with side information at the decoders. Specifically, in this model an encoder observes a pair of memoryless, arbitrarily correlated, sources $(S^n_1,S^n_2)$ and communicates with two receivers over an error-free rate-limited link of capacity $R_0$, as well as error-free rate-limited individual links of capacities $R_1$ to the first receiver and $R_2$ to the second receiver. Both receivers reproduce the source component $S^n_2$ losslessly; and Receiver $1$ also r…
Wyner's Common Information and a natural relaxation are studied in the special case of Gaussian random variables. The relaxation replaces conditional independence by a bound on the conditional mutual information. The main contribution is the proof that Gaussian auxiliaries are optimal, leading to a closed-form formula. As a corollary, the proof technique also establishes the optimality of Gaussian auxiliaries for the Gaussian Gray-Wyner network, a long-standing open problem.
An important notion of common information between two random variables is due to Wyner. In this paper, we derive a lower bound on Wyner's common information for continuous random variables. The new bound improves on the only other general lower bound on Wyner's common information, which is the mutual information. We also show that the new lower bound is tight for the so-called "Gaussian channels" case, namely, when the joint distribution of the random variables can be written as the sum of a single underlying random variable and Gaussian noises. We motivate this work from the recent variations…
The classical Gray and Wyner source coding for a simple network for sources that generate a tuple of multivariate, correlated Gaussian random variables $(Y_1,Y_2)$ is re-examined using the geometric approach of Gaussian random variables, and the weak stochastic realization of correlated Gaussian random variables. New results are: (1) The formulation, methods and algorithms to parametrize all random variables $W : Ω\rightarrow {\mathbb R}^n $ which make the two components of the tuple $(Y_1,Y_2)$ conditionally independent, according to the weak stochastic realization of $(Y_1, Y_2)$. (2) The tr…
Wyner's common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, hence also confined to, several existing interpretations in various source coding problems. This paper attempts to both generalize its definition and to expand its practical significance by providing a new operational interpretation. The generalization is two-folded: the number of dependent variables can be arbitrary, so are the alphabet of those random variables. New properties are determined for the generalized Wyner's common information of N dependen…
We consider the problem of source coding with receiver side information for the simple network proposed by R. Gray and A. Wyner in 1974. In this network, a transmitter must reliably transport the output of two correlated information sources to two receivers using three noiseless channels: a public channel which connects the transmitter to both receivers, and two private channels which connect the transmitter directly to each receiver. We extend Gray and Wyner's original problem by permitting side information to be present at each receiver. We derive inner and outer bounds for the achievable ra…
We presented assisted common information as a generalization of Gács-Körner (GK) common information at ISIT 2010. The motivation for our formulation was to improve upperbounds on the efficiency of protocols for secure two-party sampling (which is a form of secure multi-party computation). Our upperbound was based on a monotonicity property of a rate-region (called the assisted residual information region) associated with the assisted common information formulation. In this note we present further results. We explore the connection of assisted common information with the Gray-Wyner system. We s…
This paper generalizes Wyner's definition of common information of a pair of random variables to that of $N$ random variables. We prove coding theorems that show the same operational meanings for the common information of two random variables generalize to that of $N$ random variables. As a byproduct of our proof, we show that the Gray-Wyner source coding network can be generalized to $N$ source squences with $N$ decoders. We also establish a monotone property of Wyner's common information which is in contrast to other notions of the common information, specifically Shannon's mutual informatio…
The Gray and Wyner lossy source coding for a simple network for sources that generate a tuple of jointly Gaussian random variables (RVs) $X_1 : Ω\rightarrow {\mathbb R}^{p_1}$ and $X_2 : Ω\rightarrow {\mathbb R}^{p_2}$, with respect to square-error distortion at the two decoders is re-examined using (1) Hotelling's geometric approach of Gaussian RVs-the canonical variable form, and (2) van Putten's and van Schuppen's parametrization of joint distributions ${\bf P}_{X_1, X_2, W}$ by Gaussian RVs $W : Ω\rightarrow {\mathbb R}^n $ which make $(X_1,X_2)$ conditionally independent, and the weak sto…
This paper introduces a new converse machinery for a challenging class of distributed source-type problems (e.g.\ distributed source coding, common randomness generation, or hypothesis testing with communication constraints), through the example of the Wyner-Ahlswede-Körner network. Using the functional-entropic duality and the reverse hypercontractivity of the transposition semigroup, we lower bound the error probability for each joint type. Then by averaging the error probability over types, we lower bound the $c$-dispersion (which characterizes the second-order behavior of the weighted sum …
Under the paradigm of caching, partial data is delivered before the actual requests of users are known. In this paper, this problem is modeled as a canonical distributed source coding problem with side information, where the side information represents the users' requests. For the single-user case, a single-letter characterization of the optimal rate region is established, and for several important special cases, closed-form solutions are given, including the scenario of uniformly distributed user requests. In this case, it is shown that the optimal caching strategy is closely related to total…
In this paper we revisit the binary hypothesis testing problem with one-sided compression. Specifically we assume that the distribution in the null hypothesis is a mixture distribution of iid components. The distribution under the alternative hypothesis is a mixture of products of either iid distributions or finite order Markov distributions with stationary transition kernels. The problem is studied under the Neyman-Pearson framework in which our main interest is the maximum error exponent of the second type of error. We derive the optimal achievable error exponent and under a further sufficie…
A distributed binary hypothesis testing problem is studied with one observer and two decision centers. Achievable type-II error exponents are derived for testing against conditional independence when the observer communicates with the two decision centers over one common and two individual noise-free bit pipes and when it communicates with them over a noisy broadcast channel (BC). The results are based on a coding and testing scheme that splits the observations into subblocks, so that transmitter and receivers can independently apply to each subblock either Gray-Wyner coordination coding with …
This paper studies the fundamental limits of caching in a network with two receivers and two files generated by a two-component discrete memoryless source with arbitrary joint distribution. Each receiver is equipped with a cache of equal capacity, and the requested files are delivered over a shared error-free broadcast link. First, a lower bound on the optimal peak rate-memory trade-off is provided. Then, in order to leverage the correlation among the library files to alleviate the load over the shared link, a two-step correlation-aware cache-aided coded multicast (CACM) scheme is proposed. Th…
The correlation among the content distributed across a cache-aided broadcast network can be exploited to reduce the delivery load on the shared wireless link. This paper considers a two-user three-file network with correlated content, and studies its fundamental limits for the worst-case demand. A class of achievable schemes based on a two-step source coding approach is proposed. Library files are first compressed using Gray-Wyner source coding, and then cached and delivered using a combination of correlation-unaware cache-aided coded multicast schemes. The second step is interesting in its ow…
This paper studies the fundamental limits of content delivery in a cache-aided broadcast network for correlated content generated by a discrete memoryless source with arbitrary joint distribution. Each receiver is equipped with a cache of equal capacity, and the requested files are delivered over a shared error-free broadcast link. A class of achievable correlation-aware schemes based on a two-step source coding approach is proposed. Library files are first compressed, and then cached and delivered using a combination of correlation-unaware multiple-request cache-aided coded multicast schemes.…
