Distributed source coding schemes are typically based on the use of channels codes as source codes. In this paper we propose a new paradigm, termed "distributed arithmetic coding", which exploits the fact that arithmetic codes are good source as well as channel codes. In particular, we propose a distributed binary arithmetic coder for Slepian-Wolf coding with decoder side information, along with a soft joint decoder. The proposed scheme provides several advantages over existing Slepian-Wolf coders, especially its good performance at small block lengths, and the ability to incorporate arbitrary source models in the encoding process, e.g. context-based statistical models. We have compared the performance of distributed arithmetic coding with turbo codes and low-density parity-check codes, and found that the proposed approach has very competitive performance.
The reliability function of variable-rate Slepian-Wolf coding is linked to the reliability function of channel coding with constant composition codes, through which computable lower and upper bounds are derived. The bounds coincide at rates close to the Slepian-Wolf limit, yielding a complete characterization of the reliability function in that rate regime. It is shown that variable-rate Slepian-Wolf codes can significantly outperform fixed-rate Slepian-Wolf codes in terms of rate-error tradeoff. The reliability function of variable-rate Slepian-Wolf coding with rate below the Slepian-Wolf lim…
We consider Slepian-Wolf code design based on LDPC (low-density parity-check) coset codes for memoryless source-side information pairs. A density evolution formula, equipped with a concentration theorem, is derived for Slepian- Wolf coding based on LDPC coset codes. As a consequence, an intimate connection between Slepian-Wolf coding and channel coding is established. Specifically we show that, under density evolution, design of binary LDPC coset codes for Slepian-Wolf coding of an arbitrary memoryless source-side information pair reduces to design of binary LDPC codes for binary-input output-…
We consider the problem of compression of two memoryless binary sources, the correlation between which is defined by a Hidden Markov Model (HMM). We propose a Decision Feedback (DF) based scheme which when used with low density parity check codes results in compression close to the Slepian Wolf limits.
We consider the problem of compression of two memoryless binary sources, the correlation between which is defined by a Hidden Markov Model (HMM). We propose a Decision Feedback (DF) based scheme which when used with low density parity check codes results in compression close to the Slepian Wolf limits.
The polarization phenomenon for a single source is extended to a framework with multiple correlated sources. It is shown in addition to extracting the randomness of the source, the polar transforms takes the original arbitrary dependencies to extremal dependencies. Polar coding schemes for the Slepian-Wolf problem and for secret key generations are then proposed based on this phenomenon. In particular, constructions of secret keys achieving the secrecy capacity and compression schemes achieving the Slepian-Wolf capacity region are obtained with a complexity of $O(n \log (n))$.
The Slepian-Wolf bound on the admissible coding rate forms the most fundamental aspect of distributed source coding. As such, it is necessary to provide a framework with which to model more practical scenarios with respect to the arrangement of nodes in order to make Slepian-Wolf coding more suitable for multi-node Wireless Sensor Networks. This paper provides two practical scenarios in order to achieve this aim. The first is by grouping the nodes based on correlation while the second involves simplifying the structure using Markov correlation. It is found that although the bounds of these sce…
[Draft] In this paper, the redundancy of Slepian Wolf coding is revisited. Applying the random binning and converse technique in \cite{yang}, the same results in \cite{he} are obtained with much simpler proofs. Moreover, our results reflect more details about the high-order terms of the coding rate. The redundancy is investigated for both fixed-rate and variable-rate cases. The normal approximation (or dispersion) can also be obtained with minor modification.
We provide a novel achievability proof of the Slepian-Wolf theorem for i.i.d. sources over finite alphabets. We demonstrate that random codes that are linear over the real field achieve the classical Slepian-Wolf rate-region. For finite alphabets we show that typicality decoding is equivalent to solving an integer program. Minimum entropy decoding is also shown to achieve exponentially small probability of error. The techniques used may be of independent interest for code design for a wide class of information theory problems, and for the field of compressed sensing.
A method to construct nonasymmetric distributed source coding (DSC) scheme using polar codes which can achieve any point on the dominant face of the Slepian-Wolf (SW) rate region for sources with uniform marginals is considered. In addition to nonasymmetric case, we also discuss and show explicitly how asymmetric and single source compression is done using successive cancellation (SC) polar decoder. We then present simulation results that exhibit the performance of the considered methods.
We give simplify the proofs of the 2 results in Marius Zimand's paper "Kolmogorov complexity version of Slepian-Wolf coding, proceedings of STOC 2017, p22--32". The first is a universal polynomial time compression algorithm: on input $\varepsilon > 0$, a number $k$ and a string $x$ it computes in polynomial time with probability $1-\varepsilon$ a program that outputs $x$ and has length $k + O(\log^2 (|x|/\varepsilon))$, provided that there exists such a program of length at most $k$. The second result, is a distributed compression algorithm, in which several parties each send some string to a …
The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Analogously, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system to establish the source coding theorem in the second- order sense. The Slepian-Wolf coding problem for correlated sources is one of typical problems in the multi-terminal i…
In this paper, we use linear codes to study zero-error Slepian-Wolf coding of a set of sources with deviation symmetry, where the sources are generalization of the Hamming sources over an arbitrary field. We extend our previous codes, Generalized Hamming Codes for Multiple Sources, to Matrix Partition Codes and use the latter to efficiently compress the target sources. We further show that every perfect or linear-optimal code is a Matrix Partition Code. We also present some conditions when Matrix Partition Codes are perfect and/or linear-optimal. Detail discussions of Matrix Partition Codes on…
The aim of this paper is to prove theorems for the Slepian-Wolf source coding and the broadcast channel coding (independent messages and no common message) based on the the notion of a stronger version of the hash property for an ensemble of functions. Since an ensemble of sparse matrices has a strong hash property, codes using sparse matrices can realize the achievable rate region. Furthermore, extensions to the multiple source coding and multiple output broadcast channel coding are investigated.
This article proposes a novel iterative algorithm based on Low Density Parity Check (LDPC) codes for compression of correlated sources at rates approaching the Slepian-Wolf bound. The setup considered in the article looks at the problem of compressing one source at a rate determined based on the knowledge of the mean source correlation at the encoder, and employing the other correlated source as side information at the decoder which decompresses the first source based on the estimates of the actual correlation. We demonstrate that depending on the extent of the actual source correlation estima…
We analyze the optimal trade-off between the error exponent and the excess-rate exponent for variable-rate Slepian-Wolf codes. In particular, we first derive upper (converse) bounds on the optimal error and excess-rate exponents, and then lower (achievable) bounds, via a simple class of variable-rate codes which assign the same rate to all source blocks of the same type class. Then, using the exponent bounds, we derive bounds on the optimal rate functions, namely, the minimal rate assigned to each type class, needed in order to achieve a given target error exponent. The resulting excess-rate e…
We consider polar codes for memoryless sources with side information and show that the blocklength, construction, encoding and decoding complexities are bounded by a polynomial of the reciprocal of the gap between the compression rate and the conditional entropy. This extends the recent results of Guruswami and Xia to a slightly more general setting, which in turn can be applied to (1) sources with non-binary alphabets, (2) key generation for discrete and Gaussian sources, and (3) Slepian-Wolf coding and multiple accessing. In each of these cases, the complexity scaling with respect to the num…
In this paper, we propose a polar coding based scheme for set reconciliation between two network nodes. The system is modeled as a well-known Slepian-Wolf setting induced by a fixed number of deletions. The set reconciliation process is divided into two phases: 1) a deletion polar code is employed to help one node to identify the possible deletion indices, which may be larger than the number of genuine deletions; 2) a lossless compression polar code is then designed to feedback those indices with minimum overhead. Our scheme can be viewed as a generalization of polar codes to some emerging net…
We consider the problem of (almost) lossless source coding of two correlated memoryless sources using separate encoders and a joint decoder, that is, Slepian-Wolf (S-W) coding. In our setting, the encoding and decoding are asynchronous, i.e., there is a certain relative delay between the two sources. Neither the source parameters nor the relative delay are known to the encoders and the decoder. Since we assume that both encoders implement standard random binning, which does not require such knowledge anyway, the focus of this work is on the decoder. Our main contribution is in proposing a univ…
We consider a distributed parameter estimation problem, in which multiple terminals send messages related to their local observations using limited rates to a fusion center who will obtain an estimate of a parameter related to observations of all terminals. It is well known that if the transmission rates are in the Slepian-Wolf region, the fusion center can fully recover all observations and hence can construct an estimator having the same performance as that of the centralized case. One natural question is whether Slepian-Wolf rates are necessary to achieve the same estimation performance as …
We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded (using codes of fixed rate) and transmitted over two independent binary memoryless symmetric channels. The capacity of each channel is characterized by a single parameter which is not known at the transmitter. The goal is to design systems that retain near-optimal performance without channel knowledge at the transmitter. It was conjectured that it may be hard to design codes that perform well for symmetric channel conditions. In this work, we present a provable capacity-achieving sequence of LDGM ensem…
The privacy amplification is a technique to distill a secret key from a random variable by a function so that the distilled key and eavesdropper's random variable are statistically independent. There are three kinds of security criteria for the key distilled by the privacy amplification: the normalized divergence criterion, which is also known as the weak security criterion, the variational distance criterion, and the divergence criterion, which is also known as the strong security criterion. As a technique to distill a secret key, it is known that the encoder of a Slepian-Wolf (the source cod…
We present sufficient conditions for multicasting a set of correlated sources over cooperative networks. We propose joint source-Wyner-Ziv encoding/sliding-window decoding scheme, in which each receiver considers an ordered partition of other nodes. Subject to this scheme, we obtain a set of feasibility constraints for each ordered partition. We consolidate the results of different ordered partitions by utilizing a result of geometrical approach to obtain the sufficient conditions. We observe that these sufficient conditions are indeed necessary conditions for Aref networks. As a consequence o…
A new finite blocklength converse for the Slepian- Wolf coding problem is presented which significantly improves on the best known converse for this problem, due to Miyake and Kanaya [2]. To obtain this converse, an extension of the linear programming (LP) based framework for finite blocklength point- to-point coding problems from [3] is employed. However, a direct application of this framework demands a complicated analysis for the Slepian-Wolf problem. An analytically simpler approach is presented wherein LP-based finite blocklength converses for this problem are synthesized from point-to-po…
The Slepian-Wolf (SW) coding system is a source coding system with two encoders and a decoder, where these encoders independently encode source sequences from two correlated sources into codewords, and the decoder reconstructs both source sequences from the codewords. In this paper, we consider the situation in which the SW coding system is asynchronous, i.e., each encoder samples a source sequence with some unknown delay. We assume that delays are unknown but maximum and minimum values of possible delays are known to encoders and the decoder. We also assume that sources are discrete stationar…
This paper deals with the problem of multicasting a set of discrete memoryless correlated sources (DMCS) over a cooperative relay network. Necessary conditions with cut-set interpretation are presented. A \emph{Joint source-Wyner-Ziv encoding/sliding window decoding} scheme is proposed, in which decoding at each receiver is done with respect to an ordered partition of other nodes. For each ordered partition a set of feasibility constraints is derived. Then, utilizing the sub-modular property of the entropy function and a novel geometrical approach, the results of different ordered partitions a…
We analyze random coding error exponents associated with erasure/list Slepian-Wolf decoding using two different methods and then compare the resulting bounds. The first method follows the well known techniques of Gallager and Forney and the second method is based on a technique of distance enumeration, or more generally, type class enumeration, which is rooted in the statistical mechanics of a disordered system that is related to the random energy model (REM). The second method is guaranteed to yield exponent functions which are at least as tight as those of the first method, and it is demonst…
In source coding, either with or without side information at the decoder, the ultimate performance can be achieved by means of random binning. Structured binning into cosets of performing channel codes has been successfully employed in practical applications. In this letter it is formally shown that various convolutional- and turbo-syndrome decoding algorithms proposed in literature lead in fact to the same estimate. An equivalent implementation is also delineated by directly tackling syndrome decoding as a maximum a posteriori probability problem and solving it by means of iterative message-p…
We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded and transmitted over two independent binary memoryless symmetric channels. Each channel capacity is assumed to be characterized by a single parameter which is not known at the transmitter. The receiver has knowledge of both the source correlation and the channel parameters. We call a system universal if it retains near-capacity performance without channel knowledge at the transmitter. Kudekar et al. recently showed that terminated low-density parity-check (LDPC) convolutional codes (a.k.a. spatially-co…
In this work we consider a quantum generalization of the task considered by Slepian and Wolf [1973] regarding distributed source compression. In our task Alice, Bob, Charlie and Reference share a joint pure state. Alice and Bob wish to send a part of their respective systems to Charlie without collaborating with each other. We give achievability bounds for this task in the one-shot setting and provide the asymptotic and i.i.d. analysis in the case when there is no side information with Charlie. Our result implies the result of Abeyesinghe, Devetak, Hayden and Winter [2009] who studied a specia…
We propose a base-station (BS) cooperation model for broadcasting a discrete memoryless source in a cellular or heterogeneous network. The model allows the receivers to use helper BSs to improve network performance, and it permits the receivers to have prior side information about the source. We establish the model's information-theoretic limits in two operational modes: In Mode 1, the helper BSs are given information about the channel codeword transmitted by the main BS, and in Mode 2 they are provided correlated side information about the source. Optimal codes for Mode 1 use \emph{hash-and-f…
In a lossless compression system with target lengths, a compressor ${\cal C}$ maps an integer $m$ and a binary string $x$ to an $m$-bit code $p$, and if $m$ is sufficiently large, a decompressor ${\cal D}$ reconstructs $x$ from $p$. We call a pair $(m,x)$ $\textit{achievable}$ for $({\cal C},{\cal D})$ if this reconstruction is successful. We introduce the notion of an optimal compressor ${\cal C}_\text{opt}$, by the following universality property: For any compressor-decompressor pair $({\cal C}, {\cal D})$, there exists a decompressor ${\cal D}'$ such that if $(m,x)$ is achievable for $({\ca…
Practical constructions of lossless distributed source codes (for the Slepian-Wolf problem) have been the subject of much investigation in the past decade. In particular, near-capacity achieving code designs based on LDPC codes have been presented for the case of two binary sources, with a binary-symmetric correlation. However, constructing practical codes for the case of non-binary sources with arbitrary correlation remains by and large open. From a practical perspective it is also interesting to consider coding schemes whose performance remains robust to uncertainties in the joint distributi…
We study the following combinatorial version of the Slepian-Wolf coding scheme. Two isolated Senders are given binary strings $X$ and $Y$ respectively; the length of each string is equal to $n$, and the Hamming distance between the strings is at most $αn$. The Senders compress their strings and communicate the results to the Receiver. Then the Receiver must reconstruct both strings $X$ and $Y$. The aim is to minimise the lengths of the transmitted messages. For an asymmetric variant of this problem (where one of the Senders transmits the input string to the Receiver without compression) with d…
Typical random codes (TRC) in a communication scenario of source coding with side information at the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In this code ensemble, the relatively small type classes of the source are deterministically partitioned into the available bins in a one-to-one manner. As a consequence, the error probability decreases dramatically. The random binning error exponent and the error exponent of the TRC are derived and proved to be equal to one another in…
This work studies point-to-point, multiple access, and random access lossless source coding in the finite-blocklength regime. In each scenario, a random coding technique is developed and used to analyze third-order coding performance. Asymptotic results include a third-order characterization of the Slepian-Wolf rate region with an improved converse that relies on a connection to composite hypothesis testing. For dependent sources, the result implies that the independent encoders used by Slepian-Wolf codes can achieve the same third-order-optimal performance as a single joint encoder. The conce…
Distributed arithmetic coding (DAC) has been shown to be effective for Slepian-Wolf coding, especially for short data blocks. In this letter, we propose to use the DAC to compress momery-correlated sources. More specifically, the correlation between sources is modeled as a hidden Markov process. Experimental results show that the performance is close to the theoretical Slepian-Wolf limit.
In this paper, we give a distributed joint source channel coding scheme for arbitrary correlated sources for arbitrary point in the Slepian-Wolf rate region, and arbitrary link capacities using LDPC codes. We consider the Slepian-Wolf setting of two sources and one destination, with one of the sources derived from the other source by some correlation model known at the decoder. Distributed encoding and separate decoding is used for the two sources. We also give a distributed source coding scheme when the source correlation has memory to achieve any point in the Slepian-Wolf rate achievable reg…
Traditionally, ensembles of Slepian-Wolf (SW) codes are defined such that every bin of each $n$-vector of each source is randomly drawn under the uniform distribution across the sets $\{0,1,\ldots,2^{nR_X}-1\}$ and $\{0,1,\ldots,2^{nR_Y}-1\}$, where $R_X$ and $R_Y$ are the coding rates of the two sources, $X$ and $Y$, respectively. In a few more recent works, where only one source, say, $X$, is compressed and the other one, $Y$, serves as side information available at the decoder, the scope is extended to variable-rate S-W (VRSW) codes, where the rate is allowed to depend on the type class of …
We analyze the dispersions of distributed lossless source coding (the Slepian-Wolf problem), the multiple-access channel and the asymmetric broadcast channel. For the two-encoder Slepian-Wolf problem, we introduce a quantity known as the entropy dispersion matrix, which is analogous to the scalar dispersions that have gained interest recently. We prove a global dispersion result that can be expressed in terms of this entropy dispersion matrix and provides intuition on the approximate rate losses at a given blocklength and error probability. To gain better intuition about the rate at which the …
This paper deals with a coding problem called complementary delivery, where messages from two correlated sources are jointly encoded and each decoder reproduces one of two messages using the other message as the side information. Both lossless and lossy universal complementary delivery coding schemes are investigated. In the lossless case, it is demonstrated that a universal complementary delivery code can be constructed by only combining two Slepian-Wolf codes. Especially, it is shown that a universal lossless complementary delivery code, for which error probability is exponentially tight, ca…
Alice and Bob are given two correlated n-bit strings x_1 and, respectively, x_2, which they want to losslessly compress and send to Zack. They can either collaborate by sharing their strings, or work separately. We show that there is no disadvantage in the second scenario: Alice and Bob, without knowing the other party's string, can achieve almost optimal compression in the sense of Kolmogorov complexity. Furthermore, compression takes polynomial time and can be made at any combination of lengths that satisfy some necessary conditions (modulo additive polylog terms). More precisely, there exis…
We consider the optimal transmission of distributed correlated discrete memoryless sources across a network with capacity constraints. We present several previously undiscussed structural properties of the set of feasible rates and transmission schemes. We extend previous results concerning the intersection of polymatroids and contrapolymatroids to characterize when all of the vertices of the Slepian-Wolf rate region are feasible for the capacity constrained network. An explicit relationship between the conditional independence relationships of the distributed sources and the number of vertice…
Communicating arbitrarily correlated sources over interference channels is considered in this paper. A sufficient condition is found for the lossless transmission of a pair of correlated sources over a discrete memoryless interference channel. With independent sources, the sufficient condition reduces to the Han-Kobayashi achievable rate region for the interference channel. For a special correlation structure (in the sense of Slepian-Wolf, 1973), the proposed region reduces to the known achievable region for interference channels with common information. A simple example is given to show that …
We consider the model of random binning and finite-temperature decoding for Slepian-Wolf codes, from a statistical-mechanical perspective. While ordinary random channel coding is intimately related to the random energy model (REM) - a statistical-mechanical model of disordered magnetic materials, it turns out that random binning (for Slepian-Wolf coding) is analogous to another, related statistical mechanical model of strong disorder, which we call the random dilution model (RDM). We use the latter analogy to characterize phase transitions pertaining to finite- temperature Slepian-Wolf decodin…
Motivated by streaming multi-view video coding and wireless sensor networks, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate deviations regime in which the rate pairs of a sequence of codes converge, along a straight line, to various points on the boundary of the Slepian-Wolf region at a speed slower than the inverse square root of the blocklength $n$, while the error probability decays subexponentially fast in $n$. Our main result focuses on directions of approaches to corner points …
We consider a setting of Slepian--Wolf coding, where the random bin of the source vector undergoes channel coding, and then decoded at the receiver, based on additional side information, correlated to the source. For a given distribution of the randomly selected channel codewords, we propose a universal decoder that depends on the statistics of neither the correlated sources nor the channel, assuming first that they are both memoryless. Exact analysis of the random-binning/random-coding error exponent of this universal decoder shows that it is the same as the one achieved by the optimal maximu…
Distributed compression is the task of compressing correlated data by several parties, each one possessing one piece of data and acting separately. The classical Slepian-Wolf theorem (D. Slepian, J. K. Wolf, IEEE Transactions on Inf. Theory, 1973) shows that if data is generated by independent draws from a joint distribution, that is by a memoryless stochastic process, then distributed compression can achieve the same compression rates as centralized compression when the parties act together. Recently, the author (M. Zimand, STOC 2017) has obtained an analogue version of the Slepian-Wolf theor…
This work is motivated by recent results of Csiszar and Narayan (IEEE Trans. on Inform. Theory, Dec. 2004), which highlight innate connections between secrecy generation by multiple terminals and multiterminal Slepian-Wolf near-lossless data compression (sans secrecy restrictions). We propose a new approach for constructing secret and private keys based on the long-known Slepian-Wolf code for sources connected by a virtual additive noise channel, due to Wyner (IEEE Trans. on Inform. Theory, Jan. 1974). Explicit procedures for such constructions, and their substantiation, are provided.
This paper considers the problem of source coding with side information at the decoder, also called Slepian-Wolf source coding scheme. In practical applications of this coding scheme, the statistical relation between the source and the side information can vary from one data transmission to another, and there is a need to adapt the coding rate depending on the current statistical relation. In this paper, we propose a novel rate-adaptive code construction based on LDPC codes for the Slepian-Wolf source coding scheme. The proposed code design method allows to optimize the code degree distributio…
We consider a multi-way relay network with an orthogonal uplink and correlated sources, and we characterise reliable communication (in the usual Shannon sense) with a single-letter expression. The characterisation is obtained using a joint source-channel random-coding argument, which is based on a combination of Wyner et al.'s "Cascaded Slepian-Wolf Source Coding" and Tuncel's "Slepian-Wolf Coding over Broadcast Channels". We prove a separation theorem for the special case of two nodes; that is, we show that a modular code architecture with separate source and channel coding functions is (asym…
Distributed source coding (DSC) addresses the compression of correlated sources without communication links among them. This paper is concerned with the Wyner-Ziv problem: coding of an information source with side information available only at the decoder in the form of a noisy version of the source. Both the theoretical analysis and code design are addressed in the framework of multi-dimensional nested lattice coding (NLC). For theoretical analysis, accurate computation of the rate-distortion function is given under the high-resolution assumption, and a new upper bound using the derivative of…
A three terminal relay system with binary erasure channel (BEC) was considered, in which a source forwarded information to a destination with a relay's "assistance". The nested LDGM (Low-density generator-matrix) -LDPC (low-density parity-check) was designed to realize Compress-and-forward (CF) at the relay. LDGM coding compressed the received signals losslessly and LDPC realized the binning for Slepian-Wolf coding. Firstly a practical coding scheme was proposed to achieve the cut-set bound on the capacity of the system, employing LDPC and Nested LDGM-LDPC codes at the source and relay respect…
We propose an approach for constructing secret and private keys based on the long-known Slepian-Wolf code, due to Wyner, for correlated sources connected by a virtual additive noise channel. Our work is motivated by results of Csiszár and Narayan which highlight innate connections between secrecy generation by multiple terminals that observe correlated source signals and Slepian-Wolf near-lossless data compression. Explicit procedures for such constructions and their substantiation are provided. The performance of low density parity check channel codes in devising a new class of secret keys is…
We consider a wireless sensors network scenario where two nodes detect correlated sources and deliver them to a central collector via a wireless link. Differently from the Slepian-Wolf approach to distributed source coding, in the proposed scenario the sensing nodes do not perform any pre-compression of the sensed data. Original data are instead independently encoded by means of low-complexity convolutional codes. The decoder performs joint decoding with the aim of exploiting the inherent correlation between the transmitted sources. Complexity at the decoder is kept low thanks to the use of an…
We provide a novel upper-bound on Witsenhausen's rate, the rate required in the zero-error analogue of the Slepian-Wolf problem; our bound is given in terms of a new information-theoretic functional defined on a certain graph. We then use the functional to give a single letter lower-bound on the error exponent for the Slepian-Wolf problem under the vanishing error probability criterion, where the decoder has full (i.e. unencoded) side information. Our exponent stems from our new encoding scheme which makes use of source distribution only through the positions of the zeros in the `channel' matr…
In the last few years, there has been a great interest in extending the information-theoretic scenario for the non-asymptotic or one-shot case, i.e., where the channel is used only once. We provide the one-shot rate region for the distributed source-coding (Slepian-Wolf) and the multiple-access channel. Our results are based on defining a novel one-shot typical set based on smooth entropies that yields the one-shot achievable rate regions while leveraging the results from the asymptotic analysis. Our results are asymptotically optimal, i.e., for the distributed source coding they yield the sam…
In this paper, we analyze the asymptotics of the normalized remaining uncertainty of a source when a compressed or hashed version of it and correlated side-information is observed. For this system, commonly known as Slepian-Wolf source coding, we establish the optimal (minimum) rate of compression of the source to ensure that the remaining uncertainties vanish. We also study the exponential rate of decay of the remaining uncertainty to zero when the rate is above the optimal rate of compression. In our study, we consider various classes of random universal hash functions. Instead of measuring …
Scalar Quantize & Forward (QF) schemes are studied for the Two-Way Relay Channel. Different QF approaches are compared in terms of rates as well as relay and decoder complexity. A coding scheme not requiring Slepian-Wolf coding at the relay is proposed and properties of the corresponding sum-rate optimization problem are presented. A numerical scheme similar to the Blahut-Arimoto algorithm is derived that guides optimized quantizer design. The results are supported by simulations.
