We address the problem of delay in an arithmetic coding system. Due to the nature of the arithmetic coding process, source sequences causing arbitrarily large encoding or decoding delays exist. This phenomena raises the question of just how large is the expected input to output delay in these systems, i.e., once a source sequence has been encoded, what is the expected number of source letters that should be further encoded to allow full decoding of that sequence. In this paper, we derive several new upper bounds on the expected delay for a memoryless source, which improve upon a known bound due to Gallager. The bounds provided are uniform in the sense of being independent of the sequence's history. In addition, we give a sufficient condition for a source to admit a bounded expected delay, which holds for a stationary ergodic Markov source of any order.
Distributed arithmetic coding (DAC) has been shown to be effective for Slepian-Wolf coding, especially for short data blocks. In this letter, we propose to use the DAC to compress momery-correlated sources. More specifically, the correlation between sources is modeled as a hidden Markov process. Experimental results show that the performance is close to the theoretical Slepian-Wolf limit.
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols (codeword) with desired target probability distribution. The set of the output codewords constitutes a codebook (or code) of a DM. Constant-composition DM (CCDM) uses arithmetic coding to efficiently encode data into codewords from a constant-composition (CC) codebook. The CC constraint limits the size of the codebook, and hence the coding rate of the CCDM. The performance of CCDM degrades with decreasing output length. To improve the performance for short transmission blocks we present a class of multi…
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols with a desired target probability distribution. Several DMs, including shell mapping and constant-composition distribution matcher (CCDM), have been successfully employed for signal shaping, e.g., in optical-fiber or 5G. The CCDM, like many other DMs, is typically implemented by arithmetic coding (AC). In this work we implement AC based DMs using finite-precision arithmetic (FPA). An analysis of the implementation shows that FPA results in a rate-loss that shrinks exponentially with the number of precis…
This introduction to arithmetic coding is divided in two parts. The first explains how and why arithmetic coding works. We start presenting it in very general terms, so that its simplicity is not lost under layers of implementation details. Next, we show some of its basic properties, which are later used in the computational techniques required for a practical implementation. In the second part, we cover the practical implementation aspects, including arithmetic operations with low precision, the subdivision of coding and modeling, and the realization of adaptive encoders. We also analyze the …
Differential pulse-code modulation (DPCM) is recently coupled with uniform scalar quantization (SQ) to improve the rate-distortion (RD) performance for the block-based quantized compressive sensing (CS) of images. In this framework, for each block's CS measurements, a prediction is generated based on the reconstructed CS measurements of the previous blocks and subtracted from measurements of the current block in the measurement domain. The resulting residual is then quantized by uniform SQ to generate the quantization index. However, the entropy coding is still required to remove the statistic…
We consider a network coding problem where the destination wants to recover the sum of the signals (Gaussian random variables or random finite field elements) at all the source nodes, but the sum must be kept secret from an eavesdropper that can wiretap on a subset of edges. This setting arises naturally in sensor networks and federated learning, where the secrecy of the sum of the signals (e.g. weights, gradients) may be desired. While the case for finite field can be solved, the case for Gaussian random variables is surprisingly difficult. We give a simple conjecture on the necessary and suf…
Optimal zero-delay coding (quantization) of $\mathbb{R}^d$-valued linearly generated Markov sources is studied under quadratic distortion. The structure and existence of deterministic and stationary coding policies that are optimal for the infinite horizon average cost (distortion) problem are established. Prior results studying the optimality of zero-delay codes for Markov sources for infinite horizons either considered finite alphabet sources or, for the $\mathbb{R}^d$-valued case, only showed the existence of deterministic and non-stationary Markov coding policies or those which are randomi…
We present a closed-form expression for the minimal delay that is achievable in a setting that combines a buffer and an erasure code, used to mitigate the packet delay variance. The erasure code is modeled according to the recent information-theoretic results on finite block length codes. Evaluations reveal that accurate knowledge of the network parameters is essential for optimal operation. Moreover, it is shown that, when the network packet delay variance is large, the buffer delay becomes negligible. Therefore, in this case the delay budget should be spent mainly on the erasure code.
We give an overview of several of the mathematical works of Gilles Lachaud and provide a historical context. This is interspersed with some personal anecdotes highlighting many facets of his personality.
The optimal zero delay coding of a finite state Markov source is considered. The existence and structure of optimal codes are studied using a stochastic control formulation. Prior results in the literature established the optimality of deterministic Markov (Walrand-Varaiya type) coding policies for finite time horizon problem, and the optimality of both deterministic nonstationary and randomized stationary policies for the infinite time horizon problem. Our main result here shows that for any irreducible and aperiodic Markov source with a finite alphabet, \emph{deterministic and stationary} Ma…
In this paper, we investigate the secure coding issue for a wiretap channel model with fixed main channel and varying wiretap channel, by assuming that legitimate parties can obtain the wiretapping channel state information (CSI) after some time delay. For the symmetric degraded delay CSI case, we present an explicit weak security scheme by constructing secure polar codes on a one-time pad chaining structure, and prove its weak security, reliability and capability of approaching the secrecy capacity of perfect CSI case with delay CSI assumption. Further for the symmetric no-degraded delay CSI …
In this paper, we introduce novel coding schemes for wireless networks with random transmission delays. These coding schemes obviate the need for synchronicity, reduce the number of transmissions and achieve the optimal rate region in the corresponding wired model for both multiple unicast and multicast cases with up to three users under the equal rate constraint. The coding schemes are presented in two phases; first, coding schemes for line, star and line-star topologies with random transmission delays are provided. Second, any general topology with multiple bidirectional unicast and multicas…
This paper extends the well-known source coding problem of multiple descriptions, in its general and basic setting, to analog source-channel coding scenarios. Encoding-decoding functions that optimally map between the (possibly continuous valued) source and the channel spaces are numerically derived. The main technical tool is a non-convex optimization method, namely, deterministic annealing, which has recently been successfully used in other mapping optimization problems. The obtained functions exhibit several interesting structural properties, map multiple source intervals to the same interv…
We use stochastic network calculus to investigate the delay performance of a multiuser MISO system with zero-forcing beamforming. First, we consider ideal assumptions with long codewords and perfect CSI at the transmitter, where we observe a strong channel hardening effect that results in very high reliability with respect to the maximum delay of the application. We then study the system under more realistic assumptions with imperfect CSI and finite blocklength channel coding. These effects lead to interference and to transmission errors, and we derive closed-form lower and upper bounds on the…
Recent years have seen renewed attention to arithmetic coding (AC). This is thanks to the use of AC for distribution matching (DM) to control the channel input distribution in probabilistic amplitude shaping. There are two main problems inherent to AC: (1) its required arithmetic precision grows linearly with the input length, and (2) high-precision multiplications and divisions are required. Here, we introduce a multiplication-free AC-based DM technique via three lookup tables (LUTs) which solves both problems above. These LUTs are used to approximate the high-precision multiplications and di…
For data streaming applications, existing solutions are not yet able to close the gap between high data rates and low delay. This work considers the problem of data streaming under mixed delay constraints over a single communication channel with delayed feedback. We propose a novel layered adaptive causal random linear network coding (LAC-RLNC) approach with forward error correction. LAC-RLNC is a variable-to-variable coding scheme, i.e., variable recovered information data at the receiver over variable short block length and rate is proposed. Specifically, for data streaming with base and enh…
The Slepian-Wolf (SW) coding system is a source coding system with two encoders and a decoder, where these encoders independently encode source sequences from two correlated sources into codewords, and the decoder reconstructs both source sequences from the codewords. In this paper, we consider the situation in which the SW coding system is asynchronous, i.e., each encoder samples a source sequence with some unknown delay. We assume that delays are unknown but maximum and minimum values of possible delays are known to encoders and the decoder. We also assume that sources are discrete stationar…
In wireless broadcast, random linear network coding (RLNC) over GF(2^L) is known to asymptotically achieve the optimal completion delay with increasing L. However, the high decoding complexity hinders the potential applicability of RLNC schemes over large GF(2^L). In this paper, a comprehensive analysis of completion delay and decoding complexity is conducted for field-based systematic RLNC schemes in wireless broadcast. In particular, we prove that the RLNC scheme over GF(2) can also asymptotically approach the optimal completion delay per packet when the packet number goes to infinity. Moreo…
Network Coding is a packet encoding technique which has recently been shown to improve network performance (by reducing delays and increasing throughput) in broadcast and multicast communications. The cost for such an improvement comes in the form of increased decoding complexity (and thus delay) at the receivers end. Before delivering the file to higher layers, the receiver should first decode those packets. In our work we consider the broadcast transmission of a large file to N wireless users. The file is segmented into a number of blocks (each containing K packets - the Coding Window Size).…
We consider a system in which two nodes take correlated measurements of a random source with time-varying and unknown statistics. The observations of the source at the first node are to be losslessly replicated with a given probability of outage at the second node, which receives data from the first node over a constant-rate errorless channel. We develop a system and associated strategies for joint distributed source coding (encoding and decoding) and transmission control in order to achieve low end-to-end delay. Slepian-Wolf coding in its traditional form cannot be applied in our scenario, si…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of the message. Commercial data compression standards make use of Huffman or arithmetic coding at some stage of the compression process. In the proposed method, like arithmetic coding entire string is mapped to an integer but is not based on fractional numbers. Unlike both arithmetic and Huffman coding no prior entropy model of the source is required. Simple intu…
We study the broadcast transmission of a single file to an arbitrary number of receivers using Random Linear Network Coding (RLNC) in a network with unreliable channels. Due to the increased computational complexity of the decoding process (especially for large files) we apply chunked RLNC (i.e. RLNC is applied within non-overlapping subsets of the file). In our work we show the optimality of the Least Received (LR) batch scheduling policy (which was introduced in our prior work) with regards to the expected file transfer completion time. Furthermore, we refine some of our earlier results, nam…
