Necessary conditions for asymptotically optimal sliding-block or stationary codes for source coding and rate-constrained simulation of memoryless sources are presented and used to motivate a design technique for trellis-encoded source coding and rate-constrained simulation. The code structure has intuitive similarities to classic random coding arguments as well as to ``fake process'' methods and alphabet-constrained methods. Experimental evidence shows that the approach provides comparable or superior performance in comparison with previously published methods on common examples, sometimes by significant margins.
We consider the problem of communication over a three user discrete memoryless interference channel ($3-$IC). The current known coding techniques for communicating over an arbitrary $3-$IC are based on message splitting, superposition coding and binning using independent and identically distributed (iid) random codebooks. In this work, we propose a new ensemble of codes - partitioned coset codes (PCC) - that possess an appropriate mix of empirical and algebraic closure properties. We develop coding techniques that exploit algebraic closure property of PCC to enable efficient communication over…
Bi-modal (respectively, multi-modal) constrained coding refers to an encoding model whereby a user input block can be mapped to two (respectively, multiple) codewords. In current storage applications, such as optical disks, multi-modal coding allows to achieve DC control, in addition to satisfying the runlength limited (RLL) constraint specified by the recording channel. In this work, a study is initiated on bi-modal fixed-length constrained encoders. Necessary and sufficient conditions are presented for the existence of such encoders for a given constraint. It is also shown that under somewha…
Huffman coding is well known to be useful in certain decision problems involving minimizing the average number of (freely chosen) queries to determine an unknown random variable. However, in problems where the queries are more constrained, the original Huffman coding no longer works. In this paper, we proposed a general model to describe such problems and two code schemes: one is Huffman-based, and the other called GBSC (Greedy Binary Separation Coding). We proved the optimality of GBSC by induction on a binary decision tree, telling us that GBSC is at least as good as Shannon coding. We then …
This paper considers the memoryless input-constrained binary erasure channel (BEC). The channel input constraint is the $(d,\infty)$-runlength limited (RLL) constraint, which mandates that any pair of successive $1$s in the input sequence be separated by at least $d$ $0$s. We consider a scenario where there is causal, noiseless feedback from the decoder. We demonstrate a simple, labelling-based, zero-error feedback coding scheme, which we prove to be feedback capacity-achieving, and, as a by-product, obtain an explicit characterization of the feedback capacity. Our proof is based on showing th…
The constrained linear representability problem (CLRP) for polymatroids determines whether there exists a polymatroid that is linear over a specified field while satisfying a collection of constraints on the rank function. Using a computer to test whether a certain rate vector is achievable with vector linear network codes for a multi-source network coding instance and whether there exists a multi-linear secret sharing scheme achieving a specified information ratio for a given secret sharing instance are shown to be special cases of CLRP. Methods for solving CLRP built from group theoretic tec…
We present a constraint-coding scheme to correct asymmetric magnitude-$1$ errors in multi-level non-volatile memories. For large numbers of such errors, the scheme is shown to deliver better correction capability compared to known alternatives, while admitting low-complexity of decoding. Our results include an algebraic formulation of the constraint, necessary and sufficient conditions for correctability, a maximum-likelihood decoder running in complexity linear in the alphabet size, and upper bounds on the probability of failing to correct $t$ errors. Besides the superior rate-correction trad…
Privacy-constrained source coding problems have become increasingly important recently, and the utility-privacy trade-off has been investigated for various systems. As pioneering work, Yamamoto (1983) found theoretical limits of the coding rate, privacy and utility in two cases; (i) both public and private information is encoded and (ii) only public information is encoded. However, the theoretical limit has not been characterized in a more general case; (iii) encoded messages consist of public information and a part of private information. Then in this paper, we characterize the trade-off rela…
We present a case for the use of codes possessing algebraic closure properties - coset codes - in developing coding techniques and characterizing achievable rate regions for generic multi-terminal channels. In particular, we consider three diverse communication scenarios - $3-$user interference channel (many-to-many), $3-$user broadcast channel (one-to-many), and multiple access with distributed states (many-to-one) - and identify non-additive examples for which coset codes are analytically proven to yield strictly larger achievable rate regions than those achievable using iid codes. On the on…
We introduce fundamental bounds on achievable cumulative rate distribution functions (CRDF) to characterize a sequential encoding process that ensures lossless or lossy reconstruction subject to an average distortion criterion using a non-causal decoder. The CRDF describes the rate resources spent sequentially to compress the sequence. We also include a security constraint that affects the set of achievable CRDF. The information leakage is defined sequentially based on the mutual information between the source and its compressed representation, as it evolves. To characterize the security const…
A general multi-terminal source code and a general multi-terminal channel code are presented. Constrained-random-number generators with sparse matrices, which are building blocks for the code construction, are used in the construction of both encoders and decoders. Achievable regions for source coding and channel coding are derived in terms of entropy functions, where the capacity region for channel coding provides an alternative to the region of [Somekh-Baruch and Verdú, ISIT2006].
We consider the problem of revealing/sharing data in an efficient and secure way via a compact representation. The representation should ensure reliable reconstruction of the desired features/attributes while still preserve privacy of the secret parts of the data. The problem is formulated as a remote lossy source coding with a privacy constraint where the remote source consists of public and secret parts. Inner and outer bounds for the optimal tradeoff region of compression rate, distortion, and privacy leakage rate are given and shown to coincide for some special cases. When specializing the…
A scenario involving a source, a channel, and a destination, where the destination is interested in {\em both} reliably reconstructing the message transmitted by the source and estimating with a fidelity criterion the state of the channel, is considered. The source knows the channel statistics, but is oblivious to the actual channel state realization. Herein it is established that a distortion constraint for channel state estimation can be reduced to an additional cost constraint on the source input distribution, in the limit of large coding block length. A newly defined capacity-distortion fu…
We consider a scenario wherein two parties Alice and Bob are provided $X_{1}^{n}$ and $X_{2}^{n}$ -- samples that are IID from a PMF $P_{X_1 X_2}$. Alice and Bob can communicate to Charles over (noiseless) communication links of rate $R_1$ and $R_2$ respectively. Their goal is to enable Charles generate samples $Y^{n}$ such that the triple $(X_{1}^{n},X_{2}^{n},Y^{n})$ has a PMF that is close, in total variation, to $\prod P_{X_1 X_2 Y}$. In addition, the three parties may posses pairwise shared common randomness at rates $C_1$ and $C_2$. We address the problem of characterizing the set of rat…
Previous work by the authors on parity-preserving fixed-length constrained encoders is extended to the variable-length case. Parity-preserving variable-length encoders are formally defined, and, to this end, Kraft conditions are developed for the parity-preserving variable-length setting. Then, a necessary and sufficient condition is presented for the existence of deterministic parity-preserving variable-length encoders for a given constraint. Examples are provided that show that there are coding ratios where parity-preserving variable-length encoders exist, while fixed-length encoders do not.
This paper studies fixed-rate randomized vector quantization under the constraint that the quantizer's output has a given fixed probability distribution. A general representation of randomized quantizers that includes the common models in the literature is introduced via appropriate mixtures of joint probability measures on the product of the source and reproduction alphabets. Using this representation and results from optimal transport theory, the existence of an optimal (minimum distortion) randomized quantizer having a given output distribution is shown under various conditions. For sources…
The pivotal storage density win achieved by solid-state devices over magnetic devices in 2015 is a result of multiple innovations in physics, architecture, and signal processing. One of the most important innovations in that regard is enabling the storage of more than one bit per cell in the Flash device, i.e., having more than two charge levels per cell. Constrained coding is used in Flash devices to increase reliability via mitigating inter-cell interference that stems from charge propagation among cells. Recently, capacity-achieving constrained codes were introduced to serve that purpose in…
In many information-theoretic channel coding problems, adding an input cost constraint to the operational setup amounts to restricting the optimization domain in the capacity formula. This paper shows that, in contrast to common belief, such a simple modification does not hold for the cost-constrained (CC) wiretap channel (WTC). The secrecy-capacity of the discrete memoryless (DM) WTC without cost constraints is described by a single auxiliary random variable. For the CC DM-WTC, however, we show that two auxiliaries are necessary to achieve capacity. Specifically, we first derive the secrecy-c…
The input-constrained binary erasure channel (BEC) with strictly causal feedback is studied. The channel input sequence must satisfy the $(0,k)$-runlength limited (RLL) constraint, i.e., no more than $k$ consecutive `$0$'s are allowed. The feedback capacity of this channel is derived for all $k\geq 1$, and is given by $$C^\mathrm{fb}_{(0,k)}(\varepsilon) = \max\frac{\overline{\varepsilon}H_2(δ_0)+\sum_{i=1}^{k-1}\left(\overline{\varepsilon}^{i+1}H_2(δ_i)\prod_{m=0}^{i-1}δ_m\right)}{1+\sum_{i=0}^{k-1}\left(\overline{\varepsilon}^{i+1} \prod_{m=0}^{i}δ_m\right)},$$ where $\varepsilon$ is the era…
The demand of two-dimensional source coding and constrained coding has been getting higher these days, but compared to the one-dimensional case, many problems have remained open as the analysis is cumbersome. A main reason for that would be because there are no graph representations discovered so far. In this paper, we focus on a two-dimensional finite type constrained system, a set of two-dimensional blocks characterized by a finite number of two-dimensional constraints, and propose its graph representation. We then show how to generate an element of the two-dimensional finite type constraine…
DNA Data storage has recently attracted much attention due to its durable preservation and extremely high information density (bits per gram) properties. In this work, we propose a hybrid coding strategy comprising of generalized constrained codes to tackle homopolymer (run-length) limit and a protograph based low-density parity-check (LDPC) code to correct asymmetric nucleotide level (i.e., A/T/C/G) substitution errors that may occur in the process of DNA sequencing. Two sequencing techniques namely, Nanopore sequencer and Illumina sequencer with their equivalent channel models and capacities…
This paper studies a Shannon-theoretic version of the generalized distribution preserving quantization problem where a stationary and memoryless source is encoded subject to a distortion constraint and the additional requirement that the reproduction also be stationary and memoryless with a given distribution. The encoder and decoder are stochastic and assumed to have access to independent common randomness. Recent work has characterized the minimum achievable coding rate at a given distortion level when unlimited common randomness is available. Here we consider the general case where the avai…
In this work, we address the question of the largest rate of linear subcodes of Reed-Muller (RM) codes, all of whose codewords respect a runlength-limited (RLL) constraint. Our interest is in the $(d,\infty)$-RLL constraint, which mandates that every pair of successive $1$s be separated by at least $d$ $0$s. Consider any sequence $\{{\mathcal{C}_m}\}_{m\geq 1}$ of RM codes with increasing blocklength, whose rates approach $R$, in the limit as the blocklength goes to infinity. We show that for any linear $(d,\infty)$-RLL subcode, $\hat{\mathcal{C}}_m$, of the code $\mathcal{C}_m$, it holds that…
This paper asks a basic question: how much training is required to beat a universal source coder? Traditionally, there have been two types of source coders: fixed, optimum coders such as Huffman coders; and universal source coders, such as Lempel-Ziv The paper considers a third type of source coders: learned coders. These are coders that are trained on data of a particular type, and then used to encode new data of that type. This is a type of coder that has recently become very popular for (lossy) image and video coding. The paper consider two criteria for performance of learned coders: the av…
This paper focuses on the encoding and decoding of Construction D' coding lattices that can be used with shaping lattices for power-constrained channels. Two encoding methods and a decoding algorithm for Construction D' lattices are given. A design of quasi-cyclic low-density parity-check (QC-LDPC) codes to form Construction D' lattices is presented. This allows construction of nested lattice codes which are good for coding, good for shaping, and have low complexity encoding and decoding. Numerical results of using $E_8$, $BW_{16}$ and Leech lattices for shaping a Construction D' lattice indic…
Designs and methods for nested lattice codes using Construction D' lattices for coding and convolutional code lattices for shaping are described. Two encoding methods and a decoding algorithm for Construction D' coding lattices that can be used with shaping lattices for power-constrained channels are given. We construct nested lattice codes with good coding properties, a high shaping gain, and low-complexity encoding and decoding. Convolutional code generator polynomials for Construction A lattices with the greatest shaping gain are given, as a result of an extensive search. It is shown that r…
