In this paper we analyze the joint rate distortion function (RDF), for a tuple of correlated sources taking values in abstract alphabet spaces (i.e., continuous) subject to two individual distortion criteria. First, we derive structural properties of the realizations of the reproduction Random Variables (RVs), which induce the corresponding optimal test channel distributions of the joint RDF. Second, we consider a tuple of correlated multivariate jointly Gaussian RVs, $X_1 : Ω\rightarrow {\mathbb R}^{p_1}, X_2 : Ω\rightarrow {\mathbb R}^{p_2}$ with two square-error fidelity criteria, and we derive additional structural properties of the optimal realizations, and use these to characterize the RDF as a convex optimization problem with respect to the parameters of the realizations. We show that the computation of the joint RDF can be performed by semidefinite programming. Further, we derive closed-form expressions of the joint RDF, such that Gray's [1] lower bounds hold with equality, and verify their consistency with the semidefinite programming computations.
The joint nonanticipative rate distortion function (NRDF) for a tuple of random processes with individual fidelity criteria is considered. Structural properties of optimal test channel distributions are derived. Further, for the application example of the joint NRDF of a tuple of jointly multivariate Gaussian Markov processes with individual square-error fidelity criteria, a realization of the reproduction processes which induces the optimal test channel distribution is derived, and the corresponding joint NRDF is characterized. The analysis of the simplest example, of a tuple of scalar correl…
The classical Gray and Wyner source coding for a simple network for sources that generate a tuple of multivariate, correlated Gaussian random variables $(Y_1,Y_2)$ is re-examined using the geometric approach of Gaussian random variables, and the weak stochastic realization of correlated Gaussian random variables. New results are: (1) The formulation, methods and algorithms to parametrize all random variables $W : Ω\rightarrow {\mathbb R}^n $ which make the two components of the tuple $(Y_1,Y_2)$ conditionally independent, according to the weak stochastic realization of $(Y_1, Y_2)$. (2) The tr…
We consider the $k$-user successive refinement problem with causal decoder side information and derive an exponential strong converse theorem. The rate-distortion region for the problem can be derived as a straightforward extension of the two-user case by Maor and Merhav (2008). We show that for any rate-distortion tuple outside the rate-distortion region of the $k$-user successive refinement problem with causal decoder side information, the joint excess-distortion probability approaches one exponentially fast. Our proof follows by judiciously adapting the recently proposed strong converse tec…
The Gray and Wyner lossy source coding for a simple network for sources that generate a tuple of jointly Gaussian random variables (RVs) $X_1 : Ω\rightarrow {\mathbb R}^{p_1}$ and $X_2 : Ω\rightarrow {\mathbb R}^{p_2}$, with respect to square-error distortion at the two decoders is re-examined using (1) Hotelling's geometric approach of Gaussian RVs-the canonical variable form, and (2) van Putten's and van Schuppen's parametrization of joint distributions ${\bf P}_{X_1, X_2, W}$ by Gaussian RVs $W : Ω\rightarrow {\mathbb R}^n $ which make $(X_1,X_2)$ conditionally independent, and the weak sto…
Examined in this paper, is the Gray and Wyner achievable lossy rate region for a tuple of correlated multivariate Gaussian random variables (RVs) $X_1 : Ω\rightarrow {\mathbb R}^{p_1}$ and $X_2 : Ω\rightarrow {\mathbb R}^{p_2}$ with respect to square-error distortions at the two decoders. It is shown that among all joint distributions induced by a triple of RVs $(X_1,X_2, W)$, such that $W : Ω\rightarrow {\mathbb W} $ is the auxiliary RV taking continuous, countable, or finite values, the Gray and Wyner achievable rate region is characterized by jointly Gaussian RVs $(X_1,X_2, W)$ such that $W…
In 1959, Rényi proposed the information dimension and the $d$-dimensional entropy to measure the information content of general random variables. This paper proposes a generalization of information dimension to stochastic processes by defining the information dimension rate as the entropy rate of the uniformly-quantized stochastic process divided by minus the logarithm of the quantizer step size $1/m$ in the limit as $m\to\infty$. It is demonstrated that the information dimension rate coincides with the rate-distortion dimension, defined as twice the rate-distortion function $R(D)$ of the stoc…
A distributed lossy compression network with $L$ encoders and a decoder is considered. Each encoder observes a source and sends a compressed version to the decoder. The decoder produces a joint reconstruction of target signals with the mean squared error distortion below a given threshold. It is assumed that the observed sources can be expressed as the sum of target signals and corruptive noises which are independently generated from two symmetric multivariate Gaussian distributions. The minimum compression rate of this network versus the distortion threshold is referred to as the rate-distort…
This paper focuses on the structural properties of test channels, of Wyner's operational information rate distortion function (RDF), $\overline{R}(Δ_X)$, of a tuple of multivariate correlated, jointly independent and identically distributed Gaussian random variables (RVs), $\{X_t, Y_t\}_{t=1}^\infty$, $X_t: Ω\rightarrow {\mathbb R}^{n_x}$, $Y_t: Ω\rightarrow {\mathbb R}^{n_y}$, with average mean-square error at the decoder, $\frac{1}{n} {\bf E}\sum_{t=1}^n||X_t - \widehat{X}_t||^2\leq Δ_X$, when $\{Y_t\}_{t=1}^\infty$ is the side information available to the decoder only. We construct optimal …
Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define th…
In hybrid digital-analog (HDA) systems, resource allocation has been utilized to achieve desired distortion performance. However, existing studies on this issue assume error-free digital transmission, which is not valid for fading channels. With time-varying channel fading, the exact channel state information is not available at the transmitter. Thus, random outage and resulting digital distortion cannot be ignored. Moreover, rate allocation should be considered in resource allocation, since it not only determines the amount of information for digital transmission and that for analog transmiss…
