Previous work by the authors on parity-preserving fixed-length constrained encoders is extended to the variable-length case. Parity-preserving variable-length encoders are formally defined, and, to this end, Kraft conditions are developed for the parity-preserving variable-length setting. Then, a necessary and sufficient condition is presented for the existence of deterministic parity-preserving variable-length encoders for a given constraint. Examples are provided that show that there are coding ratios where parity-preserving variable-length encoders exist, while fixed-length encoders do not.
We develop a general framework to prove Kraft-type inequalities for prefix-free permutation codes for source coding with various notions of permutation code and prefix. We also show that the McMillan-type converse theorem in most of these cases does not hold, and give a general form of a counterexample. Our approach is more general and works for other structures besides permutation codes. The classical Kraft inequality for prefix-free codes as well as results about permutation codes follow as corollaries of our main theorem and main counterexample.
Low-density parity-check (LDPC) codes with the parity-based approach for distributed joint source channel coding (DJSCC) with decoder side information is described in this paper. The parity-based approach is theoretical limit achievable. Different edge degree distributions are used for source variable nodes and parity variable nodes. Particularly, the codeword-averaged density evolution (CADE) is presented for asymmetrically correlated nonuniform sources over the asymmetric memoryless transmission channel. Extensive simulations show that the splitting of variable nodes can improve the coding e…
In this paper we consider the use of variable length non prefix-free codes for coding constrained sequences of symbols. We suppose to have a Markov source where some state transitions are impossible, i.e. the stochastic matrix associated with the Markov chain has some null entries. We show that classic Kraft inequality is not a necessary condition, in general, for unique decodability under the above hypothesis and we propose a relaxed necessary inequality condition. This allows, in some cases, the use of non prefix-free codes that can give very good performance, both in terms of compression an…
In this paper, we propose a comprehensive Polar coding solution that integrates reliability calculation, rate matching and parity-check coding. Judging a channel coding design from the industry's viewpoint, there are two primary concerns: (i) low-complexity implementation in application-specific integrated circuit (ASIC), and (ii) superior \& stable performance under a wide range of code lengths and rates. The former provides cost- \& power-efficiency which are vital to any commercial system; the latter ensures flexible and robust services. Our design respects both criteria. It demonstrates be…
A joint scheme for the channel coding and spectrum spreading communication system is proposed in this paper. The Bit Error Rate (BER) performance of the joint proposed scheme is evaluated and compared with the case of channel coding scheme without spectrum spreading. We employ a single Low-Density Parity Check (LDPC) code for the both channel coding and direct sequence spectrum spreading. We show that the BER performance is significantly improved while the complexity does not increase too. The philosophy behind this achievement is that the spectrum spreading techniques provide the ability of c…
The problem of efficiently training and evaluating image classifiers that can distinguish between a large number of object categories is considered. A novel metric, sharpness, is proposed which is defined as the fraction of object categories that are above a threshold accuracy. To estimate sharpness (along with a confidence value), a technique called fraction-accurate estimation is introduced which samples categories and samples instances from these categories. In addition, a technique called parity partition coding, a special type of error correcting output code, is introduced, increasing sha…
Puncturing is a well-known coding technique widely used for constructing rate-compatible codes. In this paper, we consider the problem of puncturing low-density parity-check codes and propose a new algorithm for intentional puncturing. The algorithm is based on the puncturing of untainted symbols, i.e. nodes with no punctured symbols within their neighboring set. It is shown that the algorithm proposed here performs better than previous proposals for a range of coding rates and short proportions of punctured symbols.
In this paper, we reinterprets the $(k+2,k)$ Zigzag code in coding matrix and then propose an optimal exact repair strategy for its parity nodes, whose repair disk I/O approaches a lower bound derived in this paper.
We show how real-number codes can be used to compress correlated sources and establish a new framework for distributed lossy source coding, in which we quantize compressed sources instead of compressing quantized sources. This change in the order of binning and quantization blocks makes it possible to model correlation between continuous-valued sources more realistically and compensate for the quantization error when the sources are completely correlated. We focus on the asymmetric case, i.e., lossy source coding with side information at the decoder, also known as Wyner-Ziv coding. The encodin…
Carnegie Mellon University（卡内基梅隆大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Machine learning models are becoming the primary workhorses for many applications. Production services deploy models through prediction serving systems that take in queries and return predictions by performing inference on machine learning models. In order to scale to high query rates, prediction serving systems are run on many machines in cluster settings, and thus are prone to slowdowns and failures that inflate tail latency and cause violations of strict latency targets. Current approaches to reducing tail latency are inadequate for the latency targets of prediction serving, incur high reso…
Because of their capacity-approaching performance, graph-based codes have a wide range of applications, including communications and storage. In these codes, unequal error protection (UEP) can offer performance gains with limited rate loss. Recent empirical results in magnetic recording (MR) systems show that extra protection for the parity bits of a low-density parity-check (LDPC) code via constrained coding results in significant density gains. In particular, when UEP is applied via more reliable parity bits, higher fidelity messages of parity bits are spread to all bits by message passing a…
As in classical coding theory, quantum analogues of low-density parity-check (LDPC) codes have offered good error correction performance and low decoding complexity by employing the Calderbank-Shor-Steane (CSS) construction. However, special requirements in the quantum setting severely limit the structures such quantum codes can have. While the entanglement-assisted stabilizer formalism overcomes this limitation by exploiting maximally entangled states (ebits), excessive reliance on ebits is a substantial obstacle to implementation. This paper gives necessary and sufficient conditions for the …
Since the discovery of turbo codes 20 years ago and the subsequent re-discovery of low-density parity-check codes a few years later, the field of channel coding has experienced a number of major advances. Up until that time, code designers were usually happy with performance that came within a few decibels of the Shannon Limit, primarily due to implementation complexity constraints, whereas the new coding techniques now allow performance within a small fraction of a decibel of capacity with modest encoding and decoding complexity. Due to these significant improvements, coding standards in appl…
The main goal of coding theory is to devise efficient systems to exploit the full capacity of a communication channel, thus achieving an arbitrarily small error probability. Low Density Parity Check (LDPC) codes are a family of block codes--characterised by admitting a sparse parity check matrix--with good correction capabilities. In the present paper the orbits of subspaces of a finite projective space under the action of a Singer cycle are investigated.
DNA Data storage has recently attracted much attention due to its durable preservation and extremely high information density (bits per gram) properties. In this work, we propose a hybrid coding strategy comprising of generalized constrained codes to tackle homopolymer (run-length) limit and a protograph based low-density parity-check (LDPC) code to correct asymmetric nucleotide level (i.e., A/T/C/G) substitution errors that may occur in the process of DNA sequencing. Two sequencing techniques namely, Nanopore sequencer and Illumina sequencer with their equivalent channel models and capacities…
A coding scheme based on irregular low-density parity-check (LDPC) codes is proposed to send secret messages from a source over the Gaussian wiretap channel to a destination in the presence of a wiretapper, with the restriction that the source can send only binary phase-shift keyed (BPSK) symbols. The secrecy performance of the proposed coding scheme is measured by the secret message rate through the wiretap channel as well as the equivocation rate about the message at the wiretapper. A code search procedure is suggested to obtain irregular LDPC codes that achieve good secrecy performance in s…
This paper focuses on the encoding and decoding of Construction D' coding lattices that can be used with shaping lattices for power-constrained channels. Two encoding methods and a decoding algorithm for Construction D' lattices are given. A design of quasi-cyclic low-density parity-check (QC-LDPC) codes to form Construction D' lattices is presented. This allows construction of nested lattice codes which are good for coding, good for shaping, and have low complexity encoding and decoding. Numerical results of using $E_8$, $BW_{16}$ and Leech lattices for shaping a Construction D' lattice indic…
Linear encoders with good joint spectra are suitable candidates for optimal lossless joint source-channel coding (JSCC), where the joint spectrum is a variant of the input-output complete weight distribution and is considered good if it is close to the average joint spectrum of all linear encoders (of the same coding rate). In spite of their existence, little is known on how to construct such encoders in practice. This paper is devoted to their construction. In particular, two families of linear encoders are presented and proved to have good joint spectra. The first family is derived from Gabi…
Designs and methods for nested lattice codes using Construction D' lattices for coding and convolutional code lattices for shaping are described. Two encoding methods and a decoding algorithm for Construction D' coding lattices that can be used with shaping lattices for power-constrained channels are given. We construct nested lattice codes with good coding properties, a high shaping gain, and low-complexity encoding and decoding. Convolutional code generator polynomials for Construction A lattices with the greatest shaping gain are given, as a result of an extensive search. It is shown that r…
Since the classical work of Berlekamp, McEliece and van Tilborg, it is well known that the problem of exact maximum-likelihood (ML) decoding of general linear codes is NP-hard. In this paper, we show that exact ML decoding of a classs of asymptotically good error correcting codes--expander codes, a special case of low density parity check (LDPC) codes--over binary symmetric channels (BSCs) is possible with an expected polynomial complexity. More precisely, for any bit-flipping probability, $p$, in a nontrivial range, there exists a rate region of non-zero support and a family of asymptotically…
City University of Hong Kong（香港城市大学）；Japan Institute of Science and Technology (JAIST)（日本北陆先端科学技术大学院大学）；Huawei Technologies Co., Ltd.（华为技术有限公司）
This paper investigates a large unitarily invariant system (LUIS) involving a unitarily invariant sensing matrix, an arbitrary fixed signal distribution, and forward error control (FEC) coding. Several area properties are established based on the state evolution of orthogonal approximate message passing (OAMP) in an un-coded LUIS. Under the assumptions that the state evolution for joint OAMP and FEC decoding is correct and the replica method is reliable, we analyze the achievable rate of OAMP. We prove that OAMP reaches the constrained capacity predicted by the replica method of the LUIS with …
