Theoretical analysis has long indicated that feedback improves the error exponent but not the capacity of single-user memoryless channels. Recently Polyanskiy et al. studied the benefit of variable-length feedback with termination (VLFT) codes in the non-asymptotic regime. In that work, achievability is based on an infinite length random code and decoding is attempted at every symbol. The coding rate backoff from capacity due to channel dispersion is greatly reduced with feedback, allowing capacity to be approached with surprisingly small expected latency. This paper is mainly concerned with VLFT codes based on finite-length codes and decoding attempts only at certain specified decoding times. The penalties of using a finite block-length $N$ and a sequence of specified decoding times are studied. This paper shows that properly scaling $N$ with the expected latency can achieve the same performance up to constant terms as with $N = \infty$. The penalty introduced by periodic decoding times is a linear term of the interval between decoding times and hence the performance approaches capacity as the expected latency grows if the interval between decoding times grows sub-linearly with the expected latency.
Distributed Arithmetic Coding (DAC) is an effective implementation of Slepian-Wolf coding, especially for short data blocks. To research its properties, the concept of DAC codeword distribution along proper and wrong decoding paths has been introduced. For DAC codeword distribution of equiprobable binary sources along proper decoding paths, the problem was formatted as solving a system of functional equations. However, up to now, only one closed form was obtained at rate 0.5, while in general cases, to find the closed form of DAC codeword distribution still remains a very difficult task. This …
We analyze Linear Programming (LP) decoding of graphical binary codes operating over soft-output, symmetric and log-concave channels. We show that the error-surface, separating domain of the correct decoding from domain of the erroneous decoding, is a polytope. We formulate the problem of finding the lowest-weight pseudo-codeword as a non-convex optimization (maximization of a convex function) over a polytope, with the cost function defined by the channel and the polytope defined by the structure of the code. This formulation suggests new provably convergent heuristics for finding the lowest w…
In this correspondence, we study the minimum pseudo-weight and minimum pseudo-codewords of low-density parity-check (LDPC) codes under linear programming (LP) decoding. First, we show that the lower bound of Kelly, Sridhara, Xu and Rosenthal on the pseudo-weight of a pseudo-codeword of an LDPC code with girth greater than 4 is tight if and only if this pseudo-codeword is a real multiple of a codeword. Then, we show that the lower bound of Kashyap and Vardy on the stopping distance of an LDPC code is also a lower bound on the pseudo-weight of a pseudo-codeword of this LDPC code with girth 4, an…
Distributed Arithmetic Coding (DAC) proves to be an effective implementation of Slepian-Wolf Coding (SWC), especially for short data blocks. To study the property of DAC codewords, the author has proposed the concept of DAC codeword spectrum. For equiprobable binary sources, the problem was formatted as solving a system of functional equations. Then, to calculate DAC codeword spectrum in general cases, three approximation methods have been proposed. In this paper, the author makes use of DAC codeword spectrum as a tool to answer an important question: how many (including proper and wrong) path…
A coded modulation system is considered in which nonbinary coded symbols are mapped directly to nonbinary modulation signals. It is proved that if the modulator-channel combination satisfies a particular symmetry condition, the codeword error rate performance is independent of the transmitted codeword. It is shown that this result holds for both linear-programming decoders and sum-product decoders. In particular, this provides a natural modulation mapping for nonbinary codes mapped to PSK constellations for transmission over memoryless channels such as AWGN channels or flat fading channels wit…
Lossless variable-length source coding with codeword cost is considered for general sources. The problem setting, where we impose on unequal costs on code symbols, is called the variable-length coding with codeword cost. In this problem, the infimum of average codeword cost have been determined for general sources. On the other hand, overflow probability, which is defined as the probability of codeword cost being above a threshold, have not been considered yet. In this paper, we determine the infimum of achievable threshold in the first-order sense and the second-order sense for general source…
A novel recursive list decoding (RLD) algorithm for Reed-Muller (RM) codes based on successive permutations (SP) of the codeword is presented. A low-complexity SP scheme applied to a subset of the symmetry group of RM codes is first proposed to carefully select a good codeword permutation on the fly. Then, the proposed SP technique is integrated into an improved RLD algorithm that initializes different decoding paths with random codeword permutations, which are sampled from the full symmetry group of RM codes. Finally, efficient latency and complexity reduction schemes are introduced that virt…
This paper presents prefix codes which minimize various criteria constructed as a convex combination of maximum codeword length and average codeword length or maximum redundancy and average redundancy, including a convex combination of the average of an exponential function of the codeword length and the average redundancy. This framework encompasses as a special case several criteria previously investigated in the literature, while relations to universal coding is discussed. The coding algorithm derived is parametric resulting in re-adjusting the initial source probabilities via a weighted pr…
We consider the problem of slotted asynchronous coded communication, where in each time frame (slot), the transmitter is either silent or transmits a codeword from a given (randomly selected) codebook. The task of the decoder is to decide whether transmission has taken place, and if so, to decode the message. We derive the optimum detection/decoding rule in the sense of the best trade-off among the probabilities of decoding error, false alarm, and misdetection. For this detection/decoding rule, we then derive single-letter characterizations of the exact exponential rates of these three probabi…
Motivated by applications of biometric identification and content identification systems, we consider the problem of random coding for channels, where each codeword undergoes lossy compression (vector quantization), and where the decoder bases its decision only on the compressed codewords and the channel output, which is in turn, the channel's response to the transmission of an original codeword, before compression. For memoryless sources and memoryless channels with finite alphabets, we propose a new universal decoder and analyze its error exponent, which improves on an earlier result by Dasa…
As far as we know, there is no decoding algorithm of any binary self-dual $[40, 20, 8]$ code except for the syndrome decoding applied to the code directly. This syndrome decoding for a binary self-dual $[40,20,8]$ code is not efficient in the sense that it cannot be done by hand due to a large syndrome table. The purpose of this paper is to give two new efficient decoding algorithms for an extremal binary doubly-even self-dual $[40,20, 8]$ code $C_{40,1}^{DE}$ by hand with the help of a Hermitian self-dual $[10,5,4]$ code $E_{10}$ over $GF(4)$. The main idea of this decoding is to project code…
Despite the NP hardness of acquiring minimum distance $d_m$ for linear codes theoretically, in this paper we propose one experimental method of finding minimum-weight codewords, the weight of which is equal to $d_m$ for LDPC codes. One existing syndrome decoding method, called serial belief propagation (BP) with ordered statistic decoding (OSD), is adapted to serve our purpose. We hold the conjecture that among many candidate error patterns in OSD reprocessing, modulo 2 addition of the lightest error pattern with one of the left error patterns may generate a light codeword. When the decoding s…
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols (codeword) with desired target probability distribution. The set of the output codewords constitutes a codebook (or code) of a DM. Constant-composition DM (CCDM) uses arithmetic coding to efficiently encode data into codewords from a constant-composition (CC) codebook. The CC constraint limits the size of the codebook, and hence the coding rate of the CCDM. The performance of CCDM degrades with decreasing output length. To improve the performance for short transmission blocks we present a class of multi…
A two-part successive syndrome-check decoding of polar codes is proposed with the first part successively refining the received codeword and the second part checking its syndrome. A new formulation of the successive-cancellation (SC) decoding algorithm is presented that allows for successively refining the received codeword by comparing the log-likelihood ratio value of a frozen bit with its predefined value. The syndrome of the refined received codeword is then checked for possible errors. In case there are no errors, the decoding process is terminated. Otherwise, the decoder continues to ref…
Cycle codes are a special case of low-density parity-check (LDPC) codes and as such can be decoded using an iterative message-passing decoding algorithm on the associated Tanner graph. The existence of pseudo-codewords is known to cause the decoding algorithm to fail in certain instances. In this paper, we draw a connection between pseudo-codewords of cycle codes and the so-called edge zeta function of the associated normal graph and show how the Newton polyhedron of the zeta function equals the fundamental cone of the code, which plays a crucial role in characterizing the performance of itera…
We present novel decoding schemes for hard and soft decision decoding of block codes using the minimal weight codewords of the dual code. The decoding schemes will be described for cyclic codes where polynomials can be used, however, the modification for non-cyclic codes is possible and straight forward. The hard decision decoding calculates syndrome polynomials which are the product of the received polynomial with dual codewords. Proper cyclic shifts of these syndrome polynomials are obtained and the non-zero positions are counted componentwise for these shifts. The values of this counting ar…
Belief Propagation (BP) and Linear Programming (LP) decodings of Low Density Parity Check (LDPC) codes are discussed. We summarize results of instanton/pseudo-codeword approach developed for analysis of the error-floor domain of the codes. Instantons are special, code and decoding specific, configurations of the channel noise contributing most to the Frame-Error-Rate (FER). Instantons are decoded into pseudo-codewords. Instanton/pseudo-codeword with the lowest weight describes the largest Signal-to-Noise-Ratio (SNR) asymptotic of FER, while the whole spectra of the low weight instantons is des…
We consider the topic of universal decoding with a decoder that does not have direct access to the codebook, but only to noisy versions of the various randomly generated codewords, a problem motivated by biometrical identification systems. Both the source that generates the original (clean) codewords, and the channel that corrupts them in generating the noisy codewords, as well as the main channel for communicating the messages, are all modeled by non-unifilar, finite-state systems (hidden Markov models). As in previous works on universal decoding, here too, the average error probability of ou…
We propose a new coding scheme, called the delayed coding (DC) scheme, for channels with insertion, deletion, and substitution (IDS) errors. The proposed scheme employs delayed encoding and non-iterative detection and decoding strategies to manage the transmission of multiple codewords in a linear code. In the DC scheme, a channel input sequence consists of subblocks of multiple codewords from the previous to current time instances. At the receiver side, the maximum a posteriori detection applies to the received sequences that contain information of the codeword at the current time instance, w…
In this paper we investigate the structure of the fundamental polytope used in the Linear Programming decoding introduced by Feldman, Karger and Wainwright. We begin by showing that for expander codes, every fractional pseudocodeword always has at least a constant fraction of non-integral bits. We then prove that for expander codes, the active set of any fractional pseudocodeword is smaller by a constant fraction than the active set of any codeword. We further exploit these geometrical properties to devise an improved decoding algorithm with the same complexity order as LP decoding that provab…
We propose a decoder for Trellis-Constrained Codes, a super-class of Turbo- and LDPC codes. Inspired by amplitude amplification from quantum computing, we attempt to amplify the relative likelihood of the most likely codeword until it stands out from all other codewords.
This paper presents lossless prefix codes optimized with respect to a pay-off criterion consisting of a convex combination of maximum codeword length and average codeword length. The optimal codeword lengths obtained are based on a new coding algorithm which transforms the initial source probability vector into a new probability vector according to a merging rule. The coding algorithm is equivalent to a partition of the source alphabet into disjoint sets on which a new transformed probability vector is defined as a function of the initial source probability vector and a scalar parameter. The p…
In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth Renyi entropy characterizes the optimal exponential moment of the codeword length.
In order to understand the performance of a code under maximum-likelihood (ML) decoding, it is crucial to know the minimal codewords. In the context of linear programming (LP) decoding, it turns out to be necessary to know the minimal pseudo-codewords. This paper studies the minimal codewords and minimal pseudo-codewords of some families of codes derived from projective and Euclidean planes. Although our numerical results are only for codes of very modest length, they suggest that these code families exhibit an interesting property. Namely, all minimal pseudo-codewords that are not multiples o…
In Linear Programming (LP) decoding of a Low-Density-Parity-Check (LDPC) code one minimizes a linear functional, with coefficients related to log-likelihood ratios, over a relaxation of the polytope spanned by the codewords \cite{03FWK}. In order to quantify LP decoding, and thus to describe performance of the error-correction scheme at moderate and large Signal-to-Noise-Ratios (SNR), it is important to study the relaxed polytope to understand better its vertexes, so-called pseudo-codewords, especially those which are neighbors of the zero codeword. In this manuscript we propose a technique to…
This letter introduces a novel channel coding design framework for short-length codewords that permits balancing the tradeoff between the bit error rate floor and waterfall region by modifying a single real-valued parameter. The proposed approach is based on combining convolutional coding with a $q$-ary linear combination and unequal energy allocation, the latter being controlled by the aforementioned parameter. EXIT charts are used to shed light on the convergence characteristics of the associated iterative decoder, which is described in terms of factor graphs. Simulation results show that th…
In this paper we study the problem of increasing the decoding success probability of random linear fountain code over GF(2) for small packet lengths used in delay-intolerant applications such as multimedia streaming. Such code over GF(2) are attractive as they have lower decoding complexity than codes over larger field size, but suffer from high transmission redundancy. In our proposed coding scheme we construct a codeword which is not a linear combination of any codewords previously transmitted to mitigate such transmission redundancy. We then note the observation that the probability of rece…
In order to understand the performance of a code under maximum-likelihood (ML) decoding, one studies the codewords, in particular the minimal codewords, and their Hamming weights. In the context of linear programming (LP) decoding, one's attention needs to be shifted to the pseudo-codewords, in particular to the minimal pseudo-codewords, and their pseudo-weights. In this paper we investigate some families of codes that have good properties under LP decoding, namely certain families of low-density parity-check (LDPC) codes that are derived from projective and Euclidean planes: we study the stru…
In this paper we use the Gröbner representation of a binary linear code $\mathcal C$ to give efficient algorithms for computing the whole set of coset leaders, denoted by $\mathrm{CL}(\mathcal C)$ and the set of leader codewords, denoted by $\mathrm L(\mathcal C)$. The first algorithm could be adapted to provide not only the Newton and the covering radius of $\mathcal C$ but also to determine the coset leader weight distribution. Moreover, providing the set of leader codewords we have a test-set for decoding by a gradient-like decoding algorithm. Another contribution of this article is the rel…
We consider decoding of binary Tanner codes using message-passing iterative decoding and linear programming (LP) decoding in MBIOS channels. We present new certificates that are based on a combinatorial characterization for local-optimality of a codeword in irregular Tanner codes with respect to any MBIOS channel. This characterization is based on a conical combination of normalized weighted subtrees in the computation trees of the Tanner graph. These subtrees may have any finite height h (even equal or greater than half of the girth of the Tanner graph). In addition, the degrees of local-code…
A convolutional code $\C$ over $\ZZ[D]$ is a $\ZZ[D]$-submodule of $\ZZN[D]$ where $\ZZ[D]$ stands for the ring of polynomials with coefficients in $\ZZ$. In this paper, we study the list decoding problem of these codes when the transmission is performed over an erasure channel, that is, we study how much information one can recover from a codeword $w\in \C$ when some of its coefficients have been erased. We do that using the $p$-adic expansion of $w$ and particular representations of the parity-check polynomial matrix of the code. From these matrix polynomial representations we recursively se…
In this paper, the linear programming (LP) decoder for binary linear codes, introduced by Feldman, et al. is extended to joint-decoding of binary-input finite-state channels. In particular, we provide a rigorous definition of LP joint-decoding pseudo-codewords (JD-PCWs) that enables evaluation of the pairwise error probability between codewords and JD-PCWs. This leads naturally to a provable upper bound on decoder failure probability. If the channel is a finite-state intersymbol interference channel, then the LP joint decoder also has the maximum-likelihood (ML) certificate property and all in…
Message-passing iterative decoders for low-density parity-check (LDPC) block codes are known to be subject to decoding failures due to so-called pseudo-codewords. These failures can cause the large signal-to-noise ratio performance of message-passing iterative decoding to be worse than that predicted by the maximum-likelihood decoding union bound. In this paper we address the pseudo-codeword problem from the convolutional-code perspective. In particular, we compare the performance of LDPC convolutional codes with that of their ``wrapped'' quasi-cyclic block versions and we show that the minimu…
This paper focuses on an improved Gaussian approximation (GA) based construction of polar codes with successive cancellation (SC) decoding over an additive white Gaussian noise (AWGN) channel. Arikan has proven that polar codes with low-complexity SC decoder can approach the channel capacity of an arbitrary symmetric binary-input discrete memoryless channel, provided that the code length is chosen large enough. Nevertheless, how to construct such codes over an AWGN channel with low computational effort has been an open problem. Compared to density evolution, the GA is known as a low complexity…
In this paper, we first introduce the extended binary representation of non-binary codes, which corresponds to a covering graph of the bipartite graph associated with the non-binary code. Then we show that non-binary codewords correspond to binary codewords of the extended representation that further satisfy some simplex-constraint: that is, bits lying over the same symbol-node of the non-binary graph must form a codeword of a simplex code. Applied to the binary erasure channel, this description leads to a binary erasure decoding algorithm of non-binary LDPC codes, whose complexity depends lin…
In this paper we consider lossless source coding for a class of sources specified by the total variational distance ball centred at a fixed nominal probability distribution. The objective is to find a minimax average length source code, where the minimizers are the codeword lengths -- real numbers for arithmetic or Shannon codes -- while the maximizers are the source distributions from the total variational distance ball. Firstly, we examine the maximization of the average codeword length by converting it into an equivalent optimization problem, and we give the optimal codeword lenghts via a w…
Universal coding of integers~(UCI) is a class of variable-length code, such that the ratio of the expected codeword length to $\max\{1,H(P)\}$ is within a constant factor, where $H(P)$ is the Shannon entropy of the decreasing probability distribution $P$. However, if we consider the ratio of the expected codeword length to $H(P)$, the ratio tends to infinity by using UCI, when $H(P)$ tends to zero. To solve this issue, this paper introduces a class of codes, termed generalized universal coding of integers~(GUCI), such that the ratio of the expected codeword length to $H(P)$ is within a constan…
We present a novel iterative decoding algorithm for Reed-Muller (RM) codes, which takes advantage of a graph representation of the code. Vertices of the considered graph correspond to codewords, with two vertices being connected by an edge if and only if the Hamming distance between the corresponding codewords equals the minimum distance of the code. The algorithm uses a greedy local search to find a node optimizing a metric, e.g. the correlation between the received vector and the corresponding codeword. In addition, the cyclic redundancy check can be used to terminate the search as soon as a…
Undetected errors are important for linear codes, which are the only type of errors after hard decision and automatic-repeat-request (ARQ), but do not receive much attention on their correction. In concatenated channel coding, suboptimal source coding and joint source-channel coding, constrains among successive codewords may be utilized to improve decoding performance. In this paper, list decoding is used to correct the undetected errors. The benefit proportion of the correction is obviously improved especially on Hamming codes and Reed-Muller codes, which achieves about 40% in some cases. But…
On the Multi-Weight Shift-Sum Iterative Decoding of Short BCH Codes
Zhengzhou University（郑州大学）；Sun Yat-sen University（中山大学）；Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程广东省实验室（珠海））；Middle East Technical University（中东技术大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Skolkovo Institute of Science and Technology (Skoltech)（斯科尔科沃科学技术研究所）
Index coding achieves bandwidth savings by jointly encoding the messages demanded by all the clients in a broadcast channel. The encoding is performed in such a way that each client can retrieve its demanded message from its side information and the broadcast codeword. In general, in order to decode its demanded message symbol, a receiver may have to observe the entire transmitted codeword. Querying or downloading the codeword symbols might involve costs to a client -- such as network utilization costs and storage requirements for the queried symbols to perform decoding. In traditional index c…
A new channel coding approach was proposed in [1] for random multiple access communication over the discrete-time memoryless channel. The coding approach allows users to choose their communication rates independently without sharing the rate information among each other or with the receiver. The receiver will either decode the message or report a collision depending on whether reliable message recovery is possible. It was shown that, asymptotically as the codeword length goes to infinity, the set of communication rates supporting reliable message recovery can be characterized by an achievable …
The sequence reconstruction problem, introduced by Levenshtein in 2001, considers a communication scenario where the sender transmits a codeword from some codebook and the receiver obtains multiple noisy reads of the codeword. The common setup assumes the codebook to be the entire space and the problem is to determine the minimum number of distinct reads that is required to reconstruct the transmitted codeword. Motivated by modern storage devices, we study a variant of the problem where the number of noisy reads $N$ is fixed. Specifically, we design reconstruction codes that reconstruct a code…
We consider the decoding of linear and array codes from errors when we are only allowed to download a part of the codeword. More specifically, suppose that we have encoded $k$ data symbols using an $(n,k)$ code with code length $n$ and dimension $k.$ During storage, some of the codeword coordinates might be corrupted by errors. We aim to recover the original data by reading the corrupted codeword with a limit on the transmitting bandwidth, namely, we can only download an $α$ proportion of the corrupted codeword. For a given $α,$ our objective is to design a code and a decoding scheme such that…
We consider ensembles of channel codes that are partitioned into bins, and focus on analysis of exact random coding error exponents associated with optimum decoding of the index of the bin to which the transmitted codeword belongs. Two main conclusions arise from this analysis: (i) for independent random selection of codewords within a given type class, the random coding exponent of optimal bin index decoding is given by the ordinary random coding exponent function, computed at the rate of the entire code, independently of the exponential rate of the size of the bin. (ii) for this ensemble of …
In this paper we focus our attention on a family of finite geometry codes, called type-I projective geometry low-density parity-check (PG-LDPC) codes, that are constructed based on the projective planes PG{2,q). In particular, we study their minimal codewords and pseudo-codewords, as it is known that these vectors characterize completely the code performance under maximum-likelihood decoding and linear programming decoding, respectively. The main results of this paper consist of upper and lower bounds on the pseudo-weight of the minimal pseudo-codewords of type-I PG-LDPC codes.
Huffman coding finds an optimal prefix code for a given probability mass function. Consider situations in which one wishes to find an optimal code with the restriction that all codewords have lengths that lie in a user-specified set of lengths (or, equivalently, no codewords have lengths that lie in a complementary set). This paper introduces a polynomial-time dynamic programming algorithm that finds optimal codes for this reserved-length prefix coding problem. This has applications to quickly encoding and decoding lossless codes. In addition, one modification of the approach solves any quasia…
Given a classical error-correcting block code, the task of quantum list decoding is to produce from any quantumly corrupted codeword a short list containing all messages whose codewords exhibit high "presence" in the quantumly corrupted codeword. Efficient quantum list decoders have been used to prove a quantum hardcore property of classical codes. However, the code rates of all known families of efficiently quantum list-decodable codes are, unfortunately, too small for other practical applications. To improve those known code rates, we prove that a specific code family of polynomially small c…
A reduced complexity sequential decoding algorithm for polar (sub)codes is described. The proposed approach relies on a decomposition of the polar (sub)code being decoded into a number of outer codes, and on-demand construction of codewords of these codes in the descending order of their probability. Construction of such codewords is implemented by fast decoding algorithms, which are available for many codes arising in the decomposition of polar codes. Further complexity reduction is achieved by taking hard decisions of the intermediate LLRs, and avoiding decoding of some outer codes. Data str…
There are numerous scenarios in source coding where not only the code length but the importance of each value should also be taken into account. Different from the traditional coding theorems, by adding the importance weights for the length of the codes, we define the average cost of the weighted codeword length as an importance-aware measure of the codes. This novel information theoretical measure generalizes the average codeword length by assigning importance weights for each symbol according to users' concerns through focusing on user's selections. With such definitions, coding theorems of …
