In this paper, we study the upper and the lower bounds on the joint source-channel coding error exponent with decoder side-information. The results in the paper are non-trivial extensions of the Csiszar's classical paper [5]. Unlike the joint source-channel coding result in [5], it is not obvious whether the lower bound and the upper bound are equivalent even if the channel coding error exponent is known. For a class of channels, including the symmetric channels, we apply a game-theoretic result to establish the existence of a saddle point and hence prove that the lower and upper bounds are the same if the channel coding error exponent is known. More interestingly, we show that encoder side-information does not increase the error exponents in this case.
This paper studies variable-length (VL) source coding of general sources with side-information. Novel one-shot coding theorems for coding with common side-information available at the encoder and the decoder and Slepian- Wolf (SW) coding (i.e., with side-information only at the decoder) are given, and then, are applied to asymptotic analyses of these coding problems. Especially, a general formula for the infimum of the coding rate asymptotically achievable by weak VL-SW coding (i.e., VL-SW coding with vanishing error probability) is derived. Further, the general formula is applied to investiga…
We consider source coding with fixed lag side information at the decoder. We focus on the special case of perfect side information with unit lag corresponding to source coding with feedforward (the dual of channel coding with feedback) introduced by Pradhan. We use this duality to develop a linear complexity algorithm which achieves the rate-distortion bound for any memoryless finite alphabet source and distortion measure.
This letter investigates the role of index coding in the capacity of AWGN broadcast channels with receiver message side information. We first show that index coding is unnecessary where there are two receivers; multiplexing coding and superposition coding are sufficient to achieve the capacity region. We next show that, for more than two receivers, multiplexing coding and superposition coding alone can be suboptimal. We give an example where these two coding schemes alone cannot achieve the capacity region, but adding index coding can. This demonstrates that, in contrast to the two-receiver ca…
In this paper, new index coding problems are studied, where each receiver has erroneous side information. Although side information is a crucial part of index coding, the existence of erroneous side information has not yet been considered. We study an index code with receivers that have erroneous side information symbols in the error-free broadcast channel, which is called an index code with side information errors (ICSIE). The encoding and decoding procedures of the ICSIE are proposed, based on the syndrome decoding. Then, we derive the bounds on the optimal codelength of the proposed index c…
Let (S1,i, S2,i), distributed according to i.i.d p(s1, s2), i = 1, 2, . . . be a memoryless, correlated partial side information sequence. In this work we study channel coding and source coding problems where the partial side information (S1, S2) is available at the encoder and the decoder, respectively, and, additionally, either the encoder's or the decoder's side information is increased by a limited-rate description of the other's partial side information. We derive six special cases of channel coding and source coding problems and we characterize the capacity and the rate-distortion functi…
This paper considers the problem of source coding with side information at the decoder, also called Slepian-Wolf source coding scheme. In practical applications of this coding scheme, the statistical relation between the source and the side information can vary from one data transmission to another, and there is a need to adapt the coding rate depending on the current statistical relation. In this paper, we propose a novel rate-adaptive code construction based on LDPC codes for the Slepian-Wolf source coding scheme. The proposed code design method allows to optimize the code degree distributio…
This paper simplifies an existing coding scheme for the two-receiver discrete memoryless broadcast channel with complementary receiver side information where there is a passive eavesdropper and individual secrecy is required. The existing coding scheme is simplified in two steps by replacing Wyner secrecy coding with Carleial-Hellman secrecy coding. The resulting simplified scheme is free from redundant message splits and random components. Not least, the simplified scheme retains the existing achievable individual secrecy rate region. Finally, its construction simplicity helps us gain additio…
A multiterminal lossy coding problem, which includes various problems such as the Wyner-Ziv problem and the complementary delivery problem as special cases, is considered. It is shown that any point in the achievable rate-distortion region can be attained even if the source statistics are not known.
A lossy source coding problem is studied in which a source encoder communicates with two decoders, one with and one without correlated side information with an additional constraint on the privacy of the side information at the uninformed decoder. Two cases of this problem arise depending on the availability of the side information at the encoder. The set of all feasible rate-distortion-equivocation tuples are characterized for both cases. The difference between the informed and uninformed cases and the advantages of encoder side information for enhancing privacy are highlighted for a binary s…
One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Renyi entropy and smooth max Renyi divergence. Asymptotically (in the limit of large block lengths) this region is quantified in terms of spectral-sup conditional entropy rate and spectral- sup mutual information rate. In particular, it coincides with the rate region derived in the limit of unlimited independent and identically distributed copies of the sources.
The model of a side information "vending machine" accounts for scenarios in which acquiring side information is costly and thus should be done efficiently. In this paper, the three-node cascade source coding problem is studied under the assumption that a side information vending machine is available either at the intermediate or at the end node. In both cases, a single-letter characterization of the available trade-offs among the rate, the distortions in the reconstructions at the intermediate and at the end node, and the cost in acquiring the side information are derived under given condition…
We study source coding in the presence of side information, when the system can take actions that affect the availability, quality, or nature of the side information. We begin by extending the Wyner-Ziv problem of source coding with decoder side information to the case where the decoder is allowed to choose actions affecting the side information. We then consider the setting where actions are taken by the encoder, based on its observation of the source. Actions may have costs that are commensurate with the quality of the side information they yield, and an overall per-symbol cost constraint ma…
We study error exponents for source coding with side information. Both achievable exponents and converse bounds are obtained for the following two cases: lossless source coding with coded information (SCCSI) and lossy source coding with full side information (Wyner-Ziv). These results recover and extend several existing results on source-coding error exponents and are tight in some circumstances. Our bounds have a natural interpretation as a two-player game between nature and the code designer, with nature seeking to minimize the exponent and the code designer seeking to maximize it. In the Wy…
Consider the problem of source coding in networks with multiple receiving terminals, each having access to some kind of side information. In this case, standard coding techniques are either prohibitively complex to decode, or require network-source coding separation, resulting in sub-optimal transmission rates. To alleviate this problem, we offer a joint network-source coding scheme based on matrix sparsification at the code design phase, which allows the terminals to use an efficient decoding procedure (syndrome decoding using LDPC), despite the network coding throughout the network. Via a no…
We consider multi-terminal source coding with a single encoder and multiple decoders where either the encoder or the decoders can take cost constrained actions which affect the quality of the side information present at the decoders. For the scenario where decoders take actions, we characterize the rate-cost trade-off region for lossless source coding, and give an achievability scheme for lossy source coding for two decoders which is optimum for a variety of special cases of interest. For the case where the encoder takes actions, we characterize the rate-cost trade-off for a class of lossless …
The problem of lossless fixed-rate streaming coding of discrete memoryless sources with side information at the decoder is studied. A random time-varying tree-code is used to sequentially bin strings and a Stack Algorithm with a variable bias uses the side information to give a delay-universal coding system for lossless source coding with side information. The scheme is shown to give exponentially decaying probability of error with delay, with exponent equal to Gallager's random coding exponent for sources with side information. The mean of the random variable of computation for the stack deco…
We consider the cascade and triangular rate-distortion problem where side information is known to the source encoder and to the first user but not to the second user. We characterize the rate-distortion region for these problems. For the quadratic Gaussian case, we show that it is sufficient to consider jointly Gaussian distributions, a fact that leads to an explicit solution.
The output of a discrete Markov source is to be encoded instantaneously by a variable-rate encoder and decoded by a finite-state decoder. Our performance measure is a linear combination of the distortion and the instantaneous rate. Structure theorems, pertaining to the encoder and next-state functions are derived for every given finite-state decoder, which can have access to side information.
The recently proposed set-up of source coding with a side information "vending machine" allows the decoder to select actions in order to control the quality of the side information. The actions can depend on the message received from the encoder and on the previously measured samples of the side information, and are cost constrained. Moreover, the final estimate of the source by the decoder is a function of the encoder's message and depends causally on the side information sequence. Previous work by Permuter and Weissman has characterized the rate-distortion-cost function in the special case i…
The problem of side-information scalable (SI-scalable) source coding is considered in this work, where the encoder constructs a progressive description, such that the receiver with high quality side information will be able to truncate the bitstream and reconstruct in the rate distortion sense, while the receiver with low quality side information will have to receive further data in order to decode. We provide inner and outer bounds for general discrete memoryless sources. The achievable region is shown to be tight for the case that either of the decoders requires a lossless reconstruction, as…
This letter investigates a new class of index coding problems. One sender broadcasts packets to multiple users, each desiring a subset, by exploiting prior knowledge of linear combinations of packets. We refer to this class of problems as index coding with coded side-information. Our aim is to characterize the minimum index code length that the sender needs to transmit to simultaneously satisfy all user requests. We show that the optimal binary vector index code length is equal to the minimum rank (minrank) of a matrix whose elements consist of the sets of desired packet indices and side- info…
This paper characterizes the second-order coding rates for lossy source coding with side information available at both the encoder and the decoder. We first provide non-asymptotic bounds for this problem and then specialize the non-asymptotic bounds for three different scenarios: discrete memoryless sources, Gaussian sources, and Markov sources. We obtain the second-order coding rates for these settings. It is interesting to observe that the second-order coding rate for Gaussian source coding with Gaussian side information available at both the encoder and the decoder is the same as that for G…
This paper discusses noisy index coding problem over Gaussian broadcast channel. We propose a technique for mapping the index coded bits to M-QAM symbols such that the receivers whose side information satisfies certain conditions get coding gain, which we call the QAM side information coding gain. We compare this with the PSK side information coding gain, which was discussed in "Index Coded PSK Modulation," arXiv:1356200, [cs.IT] 19 September 2015.
The problem of source coding with side information (SCSI) is closely related to channel coding. Therefore, existing literature focuses on using the most successful channel codes namely, LDPC codes, turbo codes, and their variants, to solve this problem assuming classical unique decoding of the underlying channel code. In this paper, in contrast to classical decoding, we have taken the list decoding approach. We show that syndrome source coding using list decoding can achieve the theoretical limit. We argue that, as opposed to channel coding, the correct sequence from the list produced by the l…
We consider lossy source coding when side information affecting the distortion measure may be available at the encoder, decoder, both, or neither. For example, such distortion side information can model reliabilities for noisy measurements, sensor calibration information, or perceptual effects like masking and sensitivity to context. When the distortion side information is statistically independent of the source, we show that in many cases (e.g, for additive or multiplicative distortion side information) there is no penalty for knowing the side information only at the encoder, and there is no …
Constrained lossy source coding and channel coding with side information problems which extend the classic Wyner-Ziv and Gel'fand-Pinsker problems are considered. Inspired by applications in sensor networking and control, we first consider lossy source coding with two-sided partial side information where the quality/availability of the side information can be influenced by a cost-constrained action sequence. A decoder reconstructs a source sequence subject to the distortion constraint, and at the same time, an encoder is additionally required to be able to estimate the decoder's reconstruction…
Dual Domain Expurgated Error Exponents for Source Coding with Side Information
Centre Tecnològic de Telecomunicacions de Catalunya (CTTC)（加泰罗尼亚电信技术中心）；Eindhoven University of Technology（埃因霍温理工大学）；University of Cambridge（剑桥大学）；Universitat Politècnica de Catalunya (UPC)（加泰罗尼亚理工大学）
The problem of characterizing the capacity region for Gaussian broadcast channels with receiver message side information appears difficult and remains open for N >= 3 receivers. This paper proposes a joint network and Gelfand-Pinsker coding method for 3-receiver cases. Using the method, we establish a unified inner bound on the capacity region of 3-receiver Gaussian broadcast channels under general message side information configuration. The achievability proof of the inner bound uses an idea of joint interference cancelation, where interference is canceled by using both dirty-paper coding at …
We present novel non-asymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include (i) the Wyner-Ahlswede-Korner (WAK) problem of almost-lossless source coding with rate-limited side-information, (ii) the Wyner-Ziv (WZ) problem of lossy source coding with side-information at the decoder and (iii) the Gel'fand-Pinsker (GP) problem of channel coding with noncausal state information available at the encoder. The bounds are proved using ideas from channel simulation and channel resolvability. Our bounds for all three problems…
We consider the problem of transmission of several distributed sources over a multiple access channel (MAC) with side information at the sources and the decoder. Source-channel separation does not hold for this channel. Sufficient conditions are provided for transmission of sources with a given distortion. The source and/or the channel could have continuous alphabets (thus Gaussian sources and Gaussian MACs are special cases). Various previous results are obtained as special cases. We also provide several good joint source-channel coding schemes for a discrete/continuous source and discrete/co…
We consider the problem of transmission of several distributed correlated sources over a multiple access channel (MAC) with side information at the sources and the decoder. Source-channel separation does not hold for this channel. Sufficient conditions are provided for transmission of sources with a given distortion. The source and/or the channel could have continuous alphabets (thus Gaussian sources and Gaussian MACs are special cases). Various previous results are obtained as special cases. We also provide several good joint source-channel coding schemes for discrete sources and discrete/con…
We introduce the idea of distortion side information, which does not directly depend on the source but instead affects the distortion measure. We show that such distortion side information is not only useful at the encoder, but that under certain conditions, knowing it at only the encoder is as good as knowing it at both encoder and decoder, and knowing it at only the decoder is useless. Thus distortion side information is a natural complement to the signal side information studied by Wyner and Ziv, which depends on the source but does not involve the distortion measure. Furthermore, when both…
We consider the problem of source coding with receiver side information for the simple network proposed by R. Gray and A. Wyner in 1974. In this network, a transmitter must reliably transport the output of two correlated information sources to two receivers using three noiseless channels: a public channel which connects the transmitter to both receivers, and two private channels which connect the transmitter directly to each receiver. We extend Gray and Wyner's original problem by permitting side information to be present at each receiver. We derive inner and outer bounds for the achievable ra…
We study the joint source channel coding problem of transmitting an analog source over a Gaussian channel in two cases - (i) the presence of interference known only to the transmitter and (ii) in the presence of side information known only to the receiver. We introduce hybrid digital analog forms of the Costa and Wyner-Ziv coding schemes. Our schemes are based on random coding arguments and are different from the nested lattice schemes by Kochman and Zamir that use dithered quantization. We also discuss superimposed digital and analog schemes for the above problems which show that there are in…
In this work, the problem of transmitting an i.i.d Gaussian source over an i.i.d Gaussian wiretap channel with an i.i.d Gaussian side information is considered. The intended receiver is assumed to have a certain minimum SNR and the eavesdropper is assumed to have a strictly lower SNR compared to the intended receiver. The objective is minimizing the distortion of source reconstruction at the intended receiver. In this work, it is shown that unlike the Gaussian wiretap channel without side information, Shannon's source-channel separation coding scheme is not optimum in the sense of achieving th…
The length of an optimal scalar linear index code of a groupcast index coding problem is equal to the minrank of its side information hypergraph. The side-information hypergraph becomes a side-information graph for a special class of groupcast index coding problems known as unicast index coding problems. The computation of minrank is an NP-hard problem. There exists a low rank matrix completion method and clique cover method to find suboptimal solutions to the index coding problem represented by a side-information graph. However, both the methods are NP-hard. The number of computations require…
We study the zero-error source coding problem in which an encoder with Side Information (SI) $g(Y)$ transmits source symbols $X$ to a decoder. The decoder has SI $Y$ and wants to recover $f(X,Y)$ where $f,g$ are deterministic. We exhibit a condition on the source distribution and $g$ that we call "pairwise shared side information", such that the optimal rate has a single-letter expression. This condition is satisfied if every pair of source symbols "share" at least one SI symbol for all output of $g$. It has a practical interpretation, as $Y$ models a request made by the encoder on an image $X…
The source-coding problem with side information at the decoder is studied subject to a constraint that the encoder---to whom the side information is unavailable---be able to compute the decoder's reconstruction sequence to within some distortion. For discrete memoryless sources and finite single-letter distortion measures, an expression is given for the minimal description rate as a function of the joint law of the source and side information and of the allowed distortions at the encoder and at the decoder. The minimal description rate is also computed for a memoryless Gaussian source with squ…
We study the problem of rate-distortion-equivocation with side-information only available at the decoder when an independent private random key is shared between the sender and the receiver. The sender compresses the sequence, and the receiver reconstructs it such that the average distortion between the source and the output is limited. The equivocation is measured at an eavesdropper that intercepts the source encoded message, utilizing side-information correlated with the source and the side-information at the decoder. We have derived the entire achievable rate-distortion-equivocation region …
In this work, the problem of transmitting an i.i.d Gaussian source over an i.i.d Gaussian wiretap channel with an i.i.d Gaussian side information available at the intended receiver is considered. The intended receiver is assumed to have a certain minimum SNR and the eavesdropper is assumed to have a strictly lower SNR, compared to the intended receiver. The objective is to minimize the distortion of source reconstruction at the intended receiver. In this work, it is shown that the source-channel separation coding scheme is optimum in the sense of achieving minimum distortion. Two hybrid digita…
Inspired by the context of compressing encrypted sources, this paper considers the general tradeoff between rate, end-to-end delay, and probability of error for lossless source coding with side-information. The notion of end-to-end delay is made precise by considering a sequential setting in which source symbols are revealed in real time and need to be reconstructed at the decoder within a certain fixed latency requirement. Upper bounds are derived on the reliability functions with delay when side-information is known only to the decoder as well as when it is also known at the encoder. When th…
This paper deals with embedded index coding problem (EICP), introduced by A. Porter and M. Wootters, which is a decentralized communication problem among users with side information. An alternate definition of the parameter minrank of an EICP, which has reduced computational complexity compared to the existing definition, is presented. A graphical representation for an EICP is given using directed bipartite graphs, called bipartite problem graph, and the side information alone is represented using an undirected bipartite graph called the side information bipartite graph. Inspired by the well-s…
We consider the problem of zero error source coding with limited feedback when side information is present at the receiver. First, we derive an achievable rate region for arbitrary joint distributions on the source and the side information. When all source pairs of source and side information symbols are observable with non-zero probability, we show that this characterization gives the entire rate region. Next, we demonstrate a class of sources for which asymptotically zero feedback suffices to achieve zero-error coding at the rate promised by the Slepian-Wolf bound for asymptotically lossless…
This paper proposes a novel achievable scheme for the index problem and applies it to the caching problem. Index coding and caching are noiseless broadcast channel problems where receivers have message side information.In the index coding problem the side information sets are fixed, while in the caching problem the side information sets correspond the cache contents, which are under the control of the system designer. The proposed index coding scheme, based on distributed source coding and non-unique decoding,is shown to strictly enlarge the rate region achievable by composite coding.The novel…
This paper investigates the problem of secure lossy source coding in the presence of an eavesdropper with arbitrary correlated side informations at the legitimate decoder (referred to as Bob) and the eavesdropper (referred to as Eve). This scenario consists of an encoder that wishes to compress a source to satisfy the desired requirements on: (i) the distortion level at Bob and (ii) the equivocation rate at Eve. It is assumed that the decoders have access to correlated sources as side information. For instance, this problem can be seen as a generalization of the well-known Wyner-Ziv problem ta…
Here we write in a unified fashion (using "R(P, Q, D)") the random coding exponents in channel coding and lossy source coding. We derive their explicit forms and show, that, for a given random codebook distribution Q, the channel decoding error exponent can be viewed as an encoding success exponent in lossy source coding, and the channel correct-decoding exponent can be viewed as an encoding failure exponent in lossy source coding. We then extend the channel exponents to arbitrary D, which corresponds for D > 0 to erasure decoding and for D < 0 to list decoding. For comparison, we also derive …
We provide a complete characterization of the rate-distortion region for the multistage successive refinement of the Wyner-Ziv source coding problem with degraded side informations at the decoder. Necessary and sufficient conditions for a source to be successively refinable along a distortion vector are subsequently derived. A source-channel separation theorem is provided when the descriptions are sent over independent channels for the multistage case. Furthermore, we introduce the notion of generalized successive refinability with multiple degraded side informations. This notion captures whet…
We consider zero-delay single-user and multi-user source coding with average distortion constraint and decoder side information. The zero-delay constraint translates into causal (sequential) encoder and decoder pairs as well as the use of instantaneous codes. For the single-user setting, we show that optimal performance is attained by time sharing at most two scalar encoder-decoder pairs, that use zero-error side information codes. Side information lookahead is shown to useless in this setting. We show that the restriction to causal encoding functions is the one that causes the performance deg…
This paper studies the problem of secure communcation over the two-receiver discrete memoryless broadcast channel with one-sided receiver side information and with a passive eavesdropper. We proposed a coding scheme which is based upon the superposition-Marton framework. Secrecy techniques such as the one-time pad, Carleial-Hellman secrecy coding and Wyner serecy coding are applied to ensure individual secrecy. This scheme is shown to be capacity achieving for some cases of the degraded broadcast channel. We also notice that one-sided receiver side information provides the advantage of rate re…
We introduce the blind index coding (BIC) problem, in which a single sender communicates distinct messages to multiple users over a shared channel. Each user has partial knowledge of each message as side information. However, unlike classic index coding, in BIC, the sender is uncertain of what side information is available to each user. In particular, the sender only knows the amount of bits in each user's side information but not its content. This problem can arise naturally in caching and wireless networks. In order to blindly exploit side information in the BIC problem, we develop a hybrid …
In this paper, we establish an interesting duality between two different quantum information-processing tasks, namely, classical source coding with quantum side information, and channel coding over c-q channels. The duality relates the optimal error exponents of these two tasks, generalizing the classical results of Ahlswede and Dueck. We establish duality both at the operational level and at the level of the entropic quantities characterizing these exponents. For the latter, the duality is given by an exact relation, whereas for the former, duality manifests itself in the following sense: an …
The model of a side information "vending machine" (VM) accounts for scenarios in which the measurement of side information sequences can be controlled via the selection of cost-constrained actions. In this paper, the three-node cascade source coding problem is studied under the assumption that a side information VM is available and the intermediate and/or at the end node of the cascade. A single-letter characterization of the achievable trade-off among the transmission rates, the distortions in the reconstructions at the intermediate and at the end node, and the cost for acquiring the side inf…
In this work, we derive the random coding error exponent for the uplink phase of a two-way relay system where physical layer network coding (PNC) is employed. The error exponent is derived for the practical (yet sub-optimum) XOR channel decoding setting. We show that the random coding error exponent under optimum (i.e., maximum likelihood) PNC channel decoding can be achieved even under the sub-optimal XOR channel decoding. The derived achievability bounds provide us with valuable insight and can be used as a benchmark for the performance of practical channel-coded PNC systems employing low co…
We consider the Cascade and Triangular rate-distortion problems where the same side information is available at the source node and User 1, and the side information available at User 2 is a degraded version of the side information at the source node and User 1. We characterize the rate-distortion region for these problems. For the Cascade setup, we showed that, at User 1, decoding and re-binning the codeword sent by the source node for User 2 is optimum. We then extend our results to the Two way Cascade and Triangular setting, where the source node is interested in lossy reconstruction of the …
We investigate the Wyner-Ziv coding in which the statistics of the principal source is known but the statistics of the channel generating the side-information is unknown except that it is in a certain class. The class consists of channels such that the distortion between the principal source and the side-information is smaller than a threshold, but channels may be neither stationary nor ergodic. In this situation, we define a new rate-distortion function as the minimum rate such that there exists a Wyner-Ziv code that is universal for every channel in the class. Then, we show an upper bound an…
