We consider the problem of Linear Programming (LP) decoding of binary linear codes. The LP excess lemma was introduced by the first author, B. Ghazi, and R. Urbanke (IEEE Trans. Inf. Th., 2014) as a technique to trade crossover probability for "LP excess" over the Binary Symmetric Channel. We generalize the LP excess lemma to discrete, binary-input, Memoryless, Symmetric and LLR-Bounded (MSB) channels. As an application, we extend a result by the first author and H. Audah (IEEE Trans. Inf. Th., 2015) on the impact of redundant checks on LP decoding to discrete MSB channels.
This paper introduces algorithms for the successive-cancellation decoding and the successive-cancellation list decoding of binary polar source/channel codes. By using the symmetric parametrization of conditional probability, we reduce both space and time complexity compared to the original algorithm introduced by Tal and Vardy.
In this paper, we characterize the decoding region of algebraic soft decoding (ASD) of Reed-Solomon (RS) codes over erasure channels and binary symmetric channel (BSC). Optimal multiplicity assignment strategies (MAS) are investigated and tight bounds are derived to show the ASD can significantly outperform conventional Berlekamp Massey (BM) decoding over these channels for a wide code rate range. The analysis technique can also be extended to other channel models, e.g., RS coded modulation over erasure channels.
This paper investigates the decoding of certain Gabidulin codes that were transmitted over a channel with space-symmetric errors. Space-symmetric errors are additive error matrices that have the property that their column and row spaces are equal. We show that for channels restricted to space-symmetric errors, with high probability errors of rank up to 2(n-k)/3 can be decoded with a Gabidulin code of length n and dimension k, using a weak-self orthogonal basis as code locators.
We consider the decoding of rank metric codes assuming the error matrix is symmetric. We prove two results. First, for rates $<1/2$ there exists a broad family of rank metric codes for which any symmetric error pattern, even of maximal rank can be corrected. Moreover, the corresponding family of decodable codes includes Gabidulin codes of rate $<1/2$. Second, for rates $>1/2$, we propose a decoder for Gabidulin codes correcting symmetric errors of rank up to $n-k$. The two mentioned decoders are deterministic and worst case.
A binary code is said to be a disjunctive list-decoding $s_L$-code (LD $s_L$-code), $s \ge 2$, $L \ge 1$, if the code is identified by the incidence matrix of a family of finite sets in which the union (or disjunctive sum) of any $s$ sets can cover not more than $L-1$ other sets of the family. In this paper, we consider a similar class of binary codes which are based on a {\em symmetric disjunctive sum} (SDS) of binary symbols. By definition, the symmetric disjunctive sum (SDS) takes values from the ternary alphabet $\{0, 1, *\}$, where the symbol~$*$ denotes "erasure". Namely: SDS is equal to…
The recently introduced polar codes constitute a breakthrough in coding theory due to their capacityachieving property. This goes hand in hand with a quasilinear construction, encoding, and successive cancellation list decoding procedures based on the Plotkin construction. The decoding algorithm can be applied with slight modifications to Reed-Muller or eBCH codes, that both achieve the capacity of erasure channels, although the list size needed for good performance grows too fast to make the decoding practical even for moderate block lengths. The key ingredient for proving the capacity-achiev…
Ternary channels can be used to model the behavior of some memory devices, where information is stored in three different levels. In this paper, error correcting coding for a ternary channel where some of the error transitions are not allowed, is considered. The resulting channel is non-symmetric, therefore classical linear codes are not optimal for this channel. We define the maximum-likelihood (ML) decoding rule for ternary codes over this channel and show that it is complex to compute, since it depends on the channel error probability. A simpler alternative decoding rule which depends only …
We discuss and analyze a list-message-passing decoder with verification for low-density parity-check (LDPC) codes on the q-ary symmetric channel (q-SC). Rather than passing messages consisting of symbol probabilities, this decoder passes lists of possible symbols and marks some lists as verified. The density evolution (DE) equations for this decoder are derived and used to compute decoding thresholds. If the maximum list size is unbounded, then we find that any capacity-achieving LDPC code for the binary erasure channel can be used to achieve capacity on the q-SC for large q. The decoding thre…
Polar coding over a class of binary discrete memoryless channels with channel knowledge at the encoder is studied. It is shown that polar codes achieve the capacity of convex and one-sided classes of symmetric channels.
Non-binary low-density parity-check codes are robust to various channel impairments. However, based on the existing decoding algorithms, the decoder implementations are expensive because of their excessive computational complexity and memory usage. Based on the combinatorial optimization, we present an approximation method for the check node processing. The simulation results demonstrate that our scheme has small performance loss over the additive white Gaussian noise channel and independent Rayleigh fading channel. Furthermore, the proposed reduced-complexity realization provides significant …
Reed-Muller (RM) codes exhibit good performance under maximum-likelihood (ML) decoding due to their highly-symmetric structure. In this paper, we explore the question of whether the code symmetry of RM codes can also be exploited to achieve near-ML performance in practice. The main idea is to apply iterative decoding to a highly-redundant parity-check (PC) matrix that contains only the minimum-weight dual codewords as rows. As examples, we consider the peeling decoder for the binary erasure channel, linear-programming and belief propagation (BP) decoding for the binary-input additive white Gau…
The capacity of symmetric, neighboring and consecutive side-information single unicast index coding problems (SNC-SUICP) with number of messages equal to the number of receivers was given by Maleki, Cadambe and Jafar. For these index coding problems, an optimal index code construction by using Vandermonde matrices was proposed. This construction requires all the side-information at the receivers to decode their wanted messages and also requires large field size. In an earlier work, we constructed binary matrices of size $m \times n (m\geq n)$ such that any $n$ adjacent rows of the matrix are l…
A framework of monomial codes is considered, which includes linear codes generated by the evaluation of certain monomials. Polar and Reed-Muller codes are the two best-known representatives of such codes and can be considered as two extreme cases. Reed-Muller codes have a large automorphism group but their low-complexity maximum likelihood decoding still remains an open problem. On the other hand, polar codes have much less symmetries but admit the efficient near-ML decoding. We study the dependency between the code symmetries and the decoding efficiency. We introduce a new family of codes, pa…
We study error bounds for linear programming decoding of regular LDPC codes. For memoryless binary-input output-symmetric channels, we prove bounds on the word error probability that are inverse doubly-exponential in the girth of the factor graph. For memoryless binary-input AWGN channel, we prove lower bounds on the threshold for regular LDPC codes whose factor graphs have logarithmic girth under LP-decoding. Specifically, we prove a lower bound of $σ=0.735$ (upper bound of $\frac{Eb}{N_0}=2.67$dB) on the threshold of $(3,6)$-regular LDPC codes whose factor graphs have logarithmic girth. Our …
In this paper, we consider a few iterative decoding schemes for the joint source-channel coding of correlated sources. Specifically, we consider the joint source-channel coding of two erasure correlated sources with transmission over different erasure channels. Our main interest is in determining whether or not various code ensembles can achieve the capacity region universally over varying channel conditions. We consider two ensembles in the class of low-density generator-matrix (LDGM) codes known as Luby-Transform (LT) codes and one ensemble of low-density parity-check (LDPC) codes. We analyz…
A decoding algorithm for $q$-ary low-density parity-check codes over the $q$-ary symmetric channel is introduced. The exchanged messages are lists of symbols from $\Fq$. A density evolution analysis for maximum list sizes $1$ and $2$ is developed. Thresholds for selected regular low-density parity-check code ensembles are computed showing gains with respect to a similar algorithm in the literature. Finite-length simulation results confirm the asymptotic analysis.
Mathematical programming is a branch of applied mathematics and has recently been used to derive new decoding approaches, challenging established but often heuristic algorithms based on iterative message passing. Concepts from mathematical programming used in the context of decoding include linear, integer, and nonlinear programming, network flows, notions of duality as well as matroid and polyhedral theory. This survey article reviews and categorizes decoding methods based on mathematical programming approaches for binary linear codes over binary-input memoryless symmetric channels.
A novel method for performing error control coding in Symmetric $α-$Stable noise environments without any prior knowledge about the value of $α$ is introduced. We use an online learning framework which employs multiple distributions to decode the received block and then combines these results based on the past performance of each individual distributions. The proposed method is also able to handle a mixture of Symmetric $α-$Stable distributed noises. Performance results in turbo coded system highlight the utility of the work.
A comparison between the joint and the successive decoding schemes for a two-link case binary Chief Executive Officer (CEO) problem is presented. We utilize the logarithmic loss as the criterion for measuring and comparing the total distortion of the decoding techniques. The binary symmetric channel (BSC) is considered as the test-channel model, which provides a theoretical rate-distortion bound. Simulation results show that the joint decoding scheme provides smaller compression rate and distortion values compared to the successive decoding scheme. On the other hand the successive decoding met…
In this paper, we consider quantized decoding of LDPC codes on the binary symmetric channel. The binary message passing algorithms, while allowing extremely fast hardware implementation, are not very attractive from the perspective of performance. More complex decoders such as the ones based on belief propagation exhibit superior performance but lead to slower decoders. The approach in this paper is to consider message passing decoders that have larger message alphabet (thereby providing performance improvement) as well as low complexity (thereby ensuring fast decoding). We propose a class of …
A noisy entropy inequality for boolean functions by Samorodnitsky is applied to binary codes. It is shown that a binary code that achieves capacity on the binary erasure channel admits optimal list size for list decoding on some binary symmetric channels (in a regime where this optimal list size is exponentially large).
Existing literature on source coding with side information (SCSI) mostly uses the state-of-the-art channel codes namely LDPC codes, turbo codes, and their variants and assume classical unique decoding. In this paper, we present an algebraic approach to SCSI based on the list decoding of the underlying channel codes. We show that the theoretical limit of SCSI can be achieved in the proposed list decoding based framework when the correlation between the source and side information is $q$-ary symmetric. We argue that, as opposed to channel coding, the correct sequence from the list produced by th…
The most common decision criteria for decoding are maximum likelihood decoding and nearest neighbor decoding. It is well-known that maximum likelihood decoding coincides with nearest neighbor decoding with respect to the Hamming metric on the binary symmetric channel. In this work we study channels and metrics for which those two criteria do and do not coincide for general codes.
In this paper we show how to attain the capacity of discrete symmetric channels with polynomial time decoding complexity by considering iterated $(U|U+V)$ constructions with Reed-Solomon code or algebraic geometry code components. These codes are decoded with a recursive computation of the {\em a posteriori} probabilities of the code symbols together with the Koetter-Vardy soft decoder used for decoding the code components in polynomial time. We show that when the number of levels of the iterated $(U|U+V)$ construction tends to infinity, we attain the capacity of any discrete symmetric channel…
The performance of maximum-likelihood (ML) decoded binary linear block codes is addressed via the derivation of tightened upper bounds on their decoding error probability. The upper bounds on the block and bit error probabilities are valid for any memoryless, binary-input and output-symmetric communication channel, and their effectiveness is exemplified for various ensembles of turbo-like codes over the AWGN channel. An expurgation of the distance spectrum of binary linear block codes further tightens the resulting upper bounds.
The performance of algebraic soft-decision decoding of Reed-Solomon codes using bit-level soft information is investigated. Optimal multiplicity assignment strategies of algebraic soft-decision decoding with infinite cost are first studied over erasure channels and the binary symmetric channel. The corresponding decoding radii are calculated in closed forms and tight bounds on the error probability are derived. The multiplicity assignment strategy and the corresponding performance analysis are then generalized to characterize the decoding region of algebraic softdecision decoding over a mixed …
We consider the dynamics of belief propagation decoding of spatially coupled Low-Density Parity-Check codes. It has been conjectured that after a short transient phase, the profile of "error probabilities" along the spatial direction of a spatially coupled code develops a uniquely-shaped wave-like solution that propagates with constant velocity v. Under this assumption, and for transmission over general Binary Memoryless Symmetric channels, we derive a formula for v. We also propose approximations that are simpler to compute and support our findings using numerical data.
We investigate the binary-symmetric parallel-relay network where there is one source, one destination, and multiple relays in parallel. We show that forwarding relays, where the relays merely transmit their received signals, achieve the capacity in two ways: with coded transmission at the source and a finite number of relays, or uncoded transmission at the source and a sufficiently large number of relays. On the other hand, decoding relays, where the relays decode the source message, re-encode, and forward it to the destination, achieve the capacity when the number of relays is small. In addit…
We analyze Linear Programming (LP) decoding of graphical binary codes operating over soft-output, symmetric and log-concave channels. We show that the error-surface, separating domain of the correct decoding from domain of the erroneous decoding, is a polytope. We formulate the problem of finding the lowest-weight pseudo-codeword as a non-convex optimization (maximization of a convex function) over a polytope, with the cost function defined by the channel and the polytope defined by the structure of the code. This formulation suggests new provably convergent heuristics for finding the lowest w…
In this work, we consider adaptive linear programming (ALP) decoding of linear codes over the finite field $\mathbb{F}_p$ of size $p$ where $p$ is a prime. In particular, we provide a general construction of valid inequalities for the codeword polytope of the so-called constant-weight embedding of a single parity-check (SPC) code over any prime field. The construction is based on classes of building blocks that are assembled to form the left-hand side of an inequality according to several rules. In the case of almost doubly-symmetric valid classes we prove that the resulting inequalities are a…
When information is to be transmitted over an unknown, possibly unreliable channel, an erasure option at the decoder is desirable. Using constant-composition random codes, we propose a generalization of Csiszar and Korner's Maximum Mutual Information decoder with erasure option for discrete memoryless channels. The new decoder is parameterized by a weighting function that is designed to optimize the fundamental tradeoff between undetected-error and erasure exponents for a compound class of channels. The class of weighting functions may be further enlarged to optimize a similar tradeoff for lis…
The protograph low-density parity-check (LDPC) codes possess many attractive properties, such as the low encoding/decoding complexity and better error floor performance, and hence have been successfully applied to different types of communication and data storage channels. In this paper,we design protograph LDPC codes for communication systems corrupted by the impulsive noise, which are modeled as additive white symmetric alpha-stable noise (AWSaSN) channels. We start by presenting a novel simulation-based protograph extrinsic information transfer (P-EXIT) analysis to derive the iterative deco…
A method for estimating the performance of low-density parity-check (LDPC) codes decoded by hard-decision iterative decoding algorithms on binary symmetric channels (BSC) is proposed. Based on the enumeration of the smallest weight error patterns that can not be all corrected by the decoder, this method estimates both the frame error rate (FER) and the bit error rate (BER) of a given LDPC code with very good precision for all crossover probabilities of practical interest. Through a number of examples, we show that the proposed method can be effectively applied to both regular and irregular LDP…
The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization on commercial off-the-shelf servers. In order to mitigate the impact of straggling decoders in this platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Transmission of a single frame is considered first, and upper bounds on the resulting frame unavailability probability as a function of the decoding latency are derived by assuming a bina…
We propose a new class of efficient decoding algorithms for Reed-Muller (RM) codes over binary-input memoryless channels. The algorithms are based on projecting the code on its cosets, recursively decoding the projected codes (which are lower-order RM codes), and aggregating the reconstructions (e.g., using majority votes). We further provide extensions of the algorithms using list-decoding. We run our algorithm for AWGN channels and Binary Symmetric Channels at the short code length ($\le 1024$) regime for a wide range of code rates. Simulation results show that in both low code rate and high…
Arıkan's polar coding, is by now a well studied technique that allows achieving the symmetric capacity of binary input memoryless channels with low complexity encoding and decoding, provided that the polar decoding architecture is used and the decoding metric is matched to the true channel. In this paper, we analyze communication rates that are achievable when the polar coding/decoding architecture is used with the decoder using an incorrect model of the channel. We define the `polar mismatched capacity' as an analogue of the classical mismatched capacity, give an expression for it, and derive…
For a given family of spatially coupled codes, we prove that the LP threshold on the BSC of the graph cover ensemble is the same as the LP threshold on the BSC of the derived spatially coupled ensemble. This result is in contrast with the fact that the BP threshold of the derived spatially coupled ensemble is believed to be larger than the BP threshold of the graph cover ensemble as noted by the work of Kudekar et al. (2011, 2012). To prove this, we establish some properties related to the dual witness for LP decoding which was introduced by Feldman et al. (2007) and simplified by Daskalakis e…
It is shown that for any binary-input discrete memoryless channel $W$ with symmetric capacity $I(W)$ and any rate $R <I(W)$, the probability of block decoding error for polar coding under successive cancellation decoding satisfies $P_e \le 2^{-N^β}$ for any $β<\frac12$ when the block-length $N$ is large enough.
We address the problem of bounding below the probability of error under maximum likelihood decoding of a binary code with a known distance distribution used on a binary symmetric channel. An improved upper bound is given for the maximum attainable exponent of this probability (the reliability function of the channel). In particular, we prove that the ``random coding exponent'' is the true value of the channel reliability for code rate $R$ in some interval immediately below the critical rate of the channel. An analogous result is obtained for the Gaussian channel.
Due to process variation in nanoscale manufacturing, there may be permanently missing connections in information processing hardware. Due to timing errors in circuits, there may be missed messages in intra-chip communications, equivalent to transiently missing connections. In this work, we investigate the performance of message-passing LDPC decoders in the presence of missing connections. We prove concentration and convergence theorems that validate the use of density evolution performance analysis. Arbitrarily small error probability is not possible with missing connections, but we find suita…
An irregular LDGM-LDPC code is studied as a sub-code of an LDPC code with some randomly \emph{punctured} output-bits. It is shown that the LDGM-LDPC codes achieve rates arbitrarily close to the channel-capacity of the binary-input symmetric-output memoryless (BISOM) channel with bounded \emph{complexity}. The measure of complexity is the average-degree (per information-bit) of the check-nodes for the factor-graph of the code. A lower-bound on the average degree of the check-nodes of the irregular LDGM-LDPC codes is obtained. The bound does not depend on the decoder used at the receiver. The st…
This paper derives an improved sphere-packing (ISP) bound for finite-length codes whose transmission takes place over symmetric memoryless channels. We first review classical results, i.e., the 1959 sphere-packing (SP59) bound of Shannon for the Gaussian channel, and the 1967 sphere-packing (SP67) bound of Shannon et al. for discrete memoryless channels. A recent improvement on the SP67 bound, as suggested by Valembois and Fossorier, is also discussed. These concepts are used for the derivation of a new lower bound on the decoding error probability (referred to as the ISP bound) which is unifo…
We consider a setting of Slepian--Wolf coding, where the random bin of the source vector undergoes channel coding, and then decoded at the receiver, based on additional side information, correlated to the source. For a given distribution of the randomly selected channel codewords, we propose a universal decoder that depends on the statistics of neither the correlated sources nor the channel, assuming first that they are both memoryless. Exact analysis of the random-binning/random-coding error exponent of this universal decoder shows that it is the same as the one achieved by the optimal maximu…
Motivated by applications of rateless coding, decision feedback, and ARQ, we study the problem of universal decoding for unknown channels, in the presence of an erasure option. Specifically, we harness the competitive minimax methodology developed in earlier studies, in order to derive a universal version of Forney's classical erasure/list decoder, which in the erasure case, optimally trades off between the probability of erasure and the probability of undetected error. The proposed universal erasure decoder guarantees universal achievability of a certain fraction $ξ$ of the optimum error expo…
The asymptotic iterative decoding performances of low-density parity-check (LDPC) codes using min-sum (MS) and sum-product (SP) decoding algorithms on memoryless binary-input output-symmetric (MBIOS) channels are analyzed in this paper. For MS decoding, the analysis is done by upper bounding the bit error probability of the root bit of a tree code by the sequence error probability of a subcode of the tree code assuming the transmission of the all-zero codeword. The result is a recursive upper bound on the bit error probability after each iteration. For SP decoding, we derive a recursively dete…
High-rate product codes (PCs) and staircase codes (SCs) are ubiquitous codes in high-speed optical communication achieving near-capacity performance on the binary symmetric channel. Their success is mostly due to very efficient iterative decoding algorithms that require very little complexity. In this paper, we extend the density evolution (DE) analysis for PCs and SCs to a channel with ternary output and ternary message passing, where the third symbol marks an erasure. We investigate the performance of a standard error-and-erasure decoder and of its simplification using DE. The proposed analy…
For information transmission a binary symmetric channel is used. There is also another noisy binary symmetric channel (feedback channel), and the transmitter observes without delay all the outputs of the forward channel via that feedback channel. The transmission of an exponential number of messages (i.e. the transmission rate is positive) is considered. The achievable decoding error exponent for such a combination of channels is investigated. It is shown that if the crossover probability of the feedback channel is less than a certain positive value, then the achievable error exponent is bette…
A protograph-based low-density parity-check (LDPC) code design technique for bandwidth-efficient coded modulation is presented. The approach jointly optimizes the LDPC code node degrees and the mapping of the coded bits to the bit-interleaved coded modulation (BICM) bit-channels. For BICM with uniform input and for BICM with probabilistic shaping, binary-input symmetric-output surrogate channels are constructed and used for code design. The constructed codes perform as good as multi-edge type codes of Zhang and Kschischang (2013). For 64-ASK with probabilistic shaping, a blocklength 64800 code…
