We show that the duality between channel capacity and data compression is retained when state information is available to the sender, to the receiver, to both, or to neither. We present a unified theory for eight special cases of channel capacity and rate distortion with state information, which also extends existing results to arbitrary pairs of independent and identically distributed (i.i.d.) correlated state information available at the sender and at the receiver, respectively. In particular, the resulting general formula for channel capacity assumes the same form as the generalized Wyner Ziv rate distortion function.
This paper derives novel results on the characterization of the the causal information rate-distortion function (IRDF) $R_{c}^{it}(D)$ for arbitrarily-distributed one-sided stationary $κ$-th order Markov source x(1),x(2),.... It is first shown that Gorbunov and Pinsker's results on the stationarity of the realizations to the causal IRDF (stated for two-sided stationary sources) do not apply to the commonly used family of asymptotic average single-letter (AASL) distortion criteria. Moreover, we show that, in general, a reconstruction sequence cannot be both jointly stationary with a one-sided s…
A randomized covering-packing duality between source and channel coding will be discussed by considering the source coding problem of coding a source with a certain distortion level and by considering a channel which communicates the source within a certain distortion level. An operational view of source-channel separation for communication with a fidelity criterion will be discussed in brief.
We propose a source/channel duality in the exponential regime, where success/failure in source coding parallels error/correctness in channel coding, and a distortion constraint becomes a log-likelihood ratio (LLR) threshold. We establish this duality by first deriving exact exponents for lossy coding of a memoryless source P, at distortion D, for a general i.i.d. codebook distribution Q, for both encoding success (R < R(P,Q,D)) and failure (R > R(P,Q,D)). We then turn to maximum likelihood (ML) decoding over a memoryless channel P with an i.i.d. input Q, and show that if we substitute P=QP, Q=…
The Wyner-Ahlswede-Körner (WAK) empirical-coordination problem where the encoders cooperate via a finite-capacity one-sided link is considered. The coordination-capacity region is derived by combining several source coding techniques, such as Wyner-Ziv (WZ) coding, binning and superposition coding. Furthermore, a semi-deterministic (SD) broadcast channel (BC) with one-sided decoder cooperation is considered. Duality principles relating the two problems are presented, and the capacity region for the SD-BC setting is derived. The direct part follows from an achievable region for a general BC tha…
Sampling above the Nyquist rate is at the heart of sigma-delta modulation, where the increase in sampling rate is translated to a reduction in the overall (mean-squared-error) reconstruction distortion. This is attained by using a feedback filter at the encoder, in conjunction with a low-pass filter at the decoder. The goal of this work is to characterize the optimal trade-off between the per-sample quantization rate and the resulting mean-squared-error distortion, under various restrictions on the feedback filter. To this end, we establish a duality relation between the performance of sigma-d…
In this paper, distributed (or multiterminal) source coding with one distortion criterion and correlated messages is considered. This problem can be also called ``Berger-Yeung problem with correlated messages''. It corresponds to the source coding part of the graph-based framework for transmission of a pair of correlated sources over the multiple-access channel (MAC) where one is lossless and the other is lossy. As a result, the achievable rate-distortion region for this problem is provided. It is an information-theoretic characterization of the rate of exponential growth (as a function of the…
It is well known that independent (separate) encoding of K correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem, where the sources are repetitions of the same source, but each description must be individually good. We observe that under mild conditions about the source and distortion measure, the rate ratio Rindependent(K)/Rjoint goes to one in the limit of small rate/high distortion. Moreover, we consider the excess rate with respect to the rate-distortion function, …
A solution to the problem of asymptotically optimum perfect universal steganography of finite memoryless sources with a passive warden is provided, which is then extended to contemplate a distortion constraint. The solution rests on the fact that Slepian's Variant I permutation coding implements first-order perfect universal steganography of finite host signals with optimum embedding rate. The duality between perfect universal steganography with asymptotically optimum embedding rate and lossless universal source coding with asymptotically optimum compression rate is evinced in practice by show…
Let (S1,i, S2,i), distributed according to i.i.d p(s1, s2), i = 1, 2, . . . be a memoryless, correlated partial side information sequence. In this work we study channel coding and source coding problems where the partial side information (S1, S2) is available at the encoder and the decoder, respectively, and, additionally, either the encoder's or the decoder's side information is increased by a limited-rate description of the other's partial side information. We derive six special cases of channel coding and source coding problems and we characterize the capacity and the rate-distortion functi…
Recently, a secrecy measure based on list-reconstruction has been proposed [2], in which a wiretapper is allowed to produce a list of $2^{mR_{L}}$ reconstruction sequences and the secrecy is measured by the minimum distortion over the entire list. In this paper, we show that this list secrecy problem is equivalent to the one with secrecy measured by a new quantity \emph{lossy-equivocation}, which is proven to be the minimum optimistic 1-achievable source coding rate (the minimum coding rate needed to reconstruct the source within target distortion with positive probability for \emph{infinitely…
We consider the lossy quantum source coding problem where the task is to compress a given quantum source below its von Neumann entropy. Inspired by the duality connections between the rate-distortion and channel coding problems in the classical setting, we propose a new formulation for the lossy quantum source coding problem. This formulation differs from the existing quantum rate-distortion theory in two aspects. Firstly, we require that the reconstruction of the compressed quantum source fulfill a global error constraint as opposed to the sample-wise local error criterion used in the standar…
Constrained lossy source coding and channel coding with side information problems which extend the classic Wyner-Ziv and Gel'fand-Pinsker problems are considered. Inspired by applications in sensor networking and control, we first consider lossy source coding with two-sided partial side information where the quality/availability of the side information can be influenced by a cost-constrained action sequence. A decoder reconstructs a source sequence subject to the distortion constraint, and at the same time, an encoder is additionally required to be able to estimate the decoder's reconstruction…
