Sequential decoding, commonly applied to substitution channels, is a sub-optimal alternative to Viterbi decoding with significantly reduced memory costs. In this work, a sequential decoder for convolutional codes over channels that are prone to insertion, deletion, and substitution errors, is described and analyzed. Our decoder expands the code trellis by a new channel-state variable, called drift state, as proposed by Davey and MacKay. A suitable decoding metric on that trellis for sequential decoding is derived, generalizing the original Fano metric. The decoder is also extended to facilitate the simultaneous decoding of multiple received sequences that arise from a single transmitted sequence. Under low-noise environments, our decoding approach reduces the decoding complexity by a couple orders of magnitude in comparison to Viterbi's algorithm, albeit at slightly higher bit error rates. An analytical method to determine the computational cutoff rate is also suggested. This analysis is supported with numerical evaluations of bit error rates and computational complexity, which are compared with respect to optimal Viterbi decoding.
We consider the `one-shot frame synchronization problem' where a decoder wants to locate a sync pattern at the output of a channel on the basis of sequential observations. We assume that the sync pattern of length N starts being emitted at a random time within some interval of size A, that characterizes the asynchronism level between the transmitter and the receiver. We show that a sequential decoder can optimally locate the sync pattern, i.e., exactly, without delay, and with probability approaching one as N tends to infinity, if and only if the asynchronism level grows as O(exp(N*k)), with k…
Performance and complexity of sequential decoding of polarization-adjusted convolutional (PAC) codes is studied. In particular, a performance and computational complexity comparison of PAC codes with 5G polar codes and convolutional codes is given. A method for bounding the complexity of sequential decoding of PAC codes is proposed.
Simulation results illustrating the performance and complexity of the sequential successive cancellation decoding algorithm are presented for the case of polar subcodes with Arikan and large kernels, as well as for extended BCH\ codes. Performance comparison with Arikan PAC and LDPC codes is provided. Furthermore, complete description of the decoding algorithm is presented.
A reduced complexity sequential decoding algorithm for polar (sub)codes is described. The proposed approach relies on a decomposition of the polar (sub)code being decoded into a number of outer codes, and on-demand construction of codewords of these codes in the descending order of their probability. Construction of such codewords is implemented by fast decoding algorithms, which are available for many codes arising in the decomposition of polar codes. Further complexity reduction is achieved by taking hard decisions of the intermediate LLRs, and avoiding decoding of some outer codes. Data str…
This paper gives new results for synchronization strings, a powerful combinatorial object that allows to efficiently deal with insertions and deletions in various communication settings: $\bullet$ We give a deterministic, linear time synchronization string construction, improving over an $O(n^5)$ time randomized construction. Independently of this work, a deterministic $O(n\log^2\log n)$ time construction was just put on arXiv by Cheng, Li, and Wu. We also give a deterministic linear time construction of an infinite synchronization string, which was not known to be computable before. Both cons…
A new score function is proposed for stack decoding of polar codes, which enables one to accurately compare paths of different lengths. The proposed score function includes bias, which reflects the average behaviour of the correct path. This enables significant complexity reduction with respect to the original stack algorithm at the expense of a negligible performance loss.
Despite the extreme error-correction performance, the amount of computation of sequential decoding of the polarization-adjusted convolutional (PAC) codes is random. In sequential decoding of convolutional codes, the computational cutoff rate denotes the region between rates whose average computational complexity of decoding is finite and those which is infinite. In this paper, by benefiting from the polarization and guessing techniques, we prove that the computational cutoff rate in sequential decoding of pre-transformed polar codes polarizes. The polarization of the computational cutoff rate …
Differential distributed space-time coding (D-DSTC) has been considered to improve both diversity and data-rate in cooperative communications in the absence of channel information. However, conventionally, it is assumed that relays are perfectly synchronized in the symbol level. In practice, this assumption is easily violated due to the distributed nature of the relay networks. This paper proposes a new differential encoding and decoding process for D-DSTC systems with two relays. The proposed method is robust against synchronization errors and does not require any channel information at the d…
In this paper, the asymptotic performance of the lattice sequential decoder for LAttice Space-Time (LAST) coded MIMO channel is analyzed. We determine the rates achievable by lattice coding and sequential decoding applied to such a channel. The diversity-multiplexing tradeoff (DMT) under lattice sequential decoding is derived as a function of its parameter---the bias term, which is critical for controlling the amount of computations required at the decoding stage. Achieving low decoding complexity requires increasing the value of the bias term. However, this is done at the expense of losing th…
Variable length codes exhibit de-synchronization problems when transmitted over noisy channels. Trellis decoding techniques based on Maximum A Posteriori (MAP) estimators are often used to minimize the error rate on the estimated sequence. If the number of symbols and/or bits transmitted are known by the decoder, termination constraints can be incorporated in the decoding process. All the paths in the trellis which do not lead to a valid sequence length are suppressed. This paper presents an analytic method to assess the expected error resilience of a VLC when trellis decoding with a sequence …
The optimal diversity-multiplexing-delay tradeoff for the multi-input multi-output (MIMO) automatic repeat request (ARQ) channel can be achieved using incremental redundancy lattice space-time codes coupled with a list decoder for joint error detection and correction. Such a decoder is based on the minimum mean-square error lattice decoding principle which is implemented using sphere decoding algorithms. However, sphere decoders suffer from high computational complexity for low-to-moderate signal-to-noise ratios, especially for large signal dimensions. In this paper, we would like to construct…
In this paper, the performance limits and the computational complexity of the lattice sequential decoder are analyzed for the unconstrained additive white Gaussian noise channel. The performance analysis available in the literature for such a channel has been studied only under the use of the minimum Euclidean distance decoder that is commonly referred to as the lattice decoder. Lattice decoders based on solutions to the NP-hard closest vector problem are very complex to implement, and the search for low complexity receivers for the detection of lattice codes is considered a challenging proble…
A sequential updating scheme (SUS) for belief propagation (BP) decoding of LDPC codes over Galois fields, $GF(q)$, and correlated Markov sources is proposed, and compared with the standard parallel updating scheme (PUS). A thorough experimental study of various transmission settings indicates that the convergence rate, in iterations, of the BP algorithm (and subsequently its complexity) for the SUS is about one half of that for the PUS, independent of the finite field size $q$. Moreover, this 1/2 factor appears regardless of the correlations of the source and the channel's noise model, while t…
This note is a written and extended version of the Shannon Lecture I gave at 2019 International Symposium on Information Theory. It gives an account of the original ideas that motivated the development of polar coding and discusses some new ideas for exploiting channel polarization more effectively in order to improve the performance of polar codes.
Differential distributed space-time coding (D-DSTC) is a cooperative transmission technique that can improve diversity in wireless relay networks in the absence of channel information. Conventionally, it is assumed that channels are flat-fading and relays are perfectly synchronized at the symbol level. However, due to the delay spread in broadband systems and the distributed nature of relay networks, these assumptions may be violated. Hence, inter-symbol interference (ISI) may appear. This paper proposes a new differential encoding and decoding process for D-DSTC systems with multiple relays o…
his study presents a novel technique to estimate the computational complexity of sequential decoding using the Berry-Esseen theorem. Unlike the theoretical bounds determined by the conventional central limit theorem argument, which often holds only for sufficiently large codeword length, the new bound obtained from the Berry-Esseen theorem is valid for any blocklength. The accuracy of the new bound is then examined for two sequential decoding algorithms, an ordering-free variant of the generalized Dijkstra's algorithm (GDA)(or simplified GDA) and the maximum-likelihood sequential decoding algo…
We consider the problem of constructing codes that can correct $δ$ deletions occurring in an arbitrary binary string of length $n$ bits. Varshamov-Tenengolts (VT) codes, dating back to 1965, are zero-error single deletion $(δ=1)$ correcting codes, and have an asymptotically optimal redundancy. Finding similar codes for $δ\geq 2$ deletions remains an open problem. In this work, we relax the standard zero-error (i.e., worst-case) decoding requirement by assuming that the positions of the $δ$ deletions (or insertions) are independent of the codeword. Our contribution is a new family of explicit c…
Motivated by video coding applications, the problem of sequential coding of correlated sources with encoding and/or decoding frame-delays is studied. The fundamental tradeoffs between individual frame rates, individual frame distortions, and encoding/decoding frame-delays are derived in terms of a single-letter information-theoretic characterization of the rate-distortion region for general inter-frame source correlations and certain types of potentially frame specific and coupled single-letter fidelity criteria. The sum-rate-distortion region is characterized in terms of generalized directed …
A common problem on sequential-type decoding is that at the signal-to-noise ratio (SNR) below the one corresponding to the cutoff rate, the average decoding complexity per information bit and the required stack size grow rapidly with the information length. In order to alleviate the problem in the maximum-likelihood sequential decoding algorithm (MLSDA), we propose to directly eliminate the top path whose end node is $Δ$-trellis-level prior to the farthest one among all nodes that have been expanded thus far by the sequential search. Following random coding argument, we analyze the early-elimi…
We study codes that are list-decodable under insertions and deletions. Specifically, we consider the setting where a codeword over some finite alphabet of size $q$ may suffer from $δ$ fraction of adversarial deletions and $γ$ fraction of adversarial insertions. A code is said to be $L$-list-decodable if there is an (efficient) algorithm that, given a received word, reports a list of $L$ codewords that include the original codeword. Using the concept of synchronization strings, introduced by the first two authors [STOC 2017], we show some surprising results. We show that for every $0\leqδ<1$, e…
Polar coding gives rise to the first explicit family of codes that provably achieve capacity with efficient encoding and decoding for a wide range of channels. However, its performance at short block lengths is far from optimal. Arikan has recently presented a new polar coding scheme, which he called polarization-adjusted convolutional (PAC) codes. Such PAC codes provide dramatic improvement in performance as compared to both standard successive-cancellation decoding as well as CRC-aided list decoding. Arikan's PAC codes are based primarily upon the following ideas: replacing CRC precoding wit…
In this paper, we study the power of sequential decoding strategies for several channels with classical input and quantum output. In our sequential decoding strategies, the receiver loops through all candidate messages trying to project the received state onto a `typical' subspace for the candidate message under consideration, stopping if the projection succeeds for a message, which is then declared as the guess of the receiver for the sent message. We show that even such a conceptually simple strategy can be used to achieve rates up to the mutual information for a single sender single receive…
We consider receiver design for coded transmission over linear Gaussian channels. We restrict ourselves to the class of lattice codes and formulate the joint detection and decoding problem as a closest lattice point search (CLPS). Here, a tree search framework for solving the CLPS is adopted. In our framework, the CLPS algorithm decomposes into the preprocessing and tree search stages. The role of the preprocessing stage is to expose the tree structure in a form {\em matched} to the search stage. We argue that the minimum mean square error decision feedback (MMSE-DFE) frontend is instrumental …
The problem of lossless fixed-rate streaming coding of discrete memoryless sources with side information at the decoder is studied. A random time-varying tree-code is used to sequentially bin strings and a Stack Algorithm with a variable bias uses the side information to give a delay-universal coding system for lossless source coding with side information. The scheme is shown to give exponentially decaying probability of error with delay, with exponent equal to Gallager's random coding exponent for sources with side information. The mean of the random variable of computation for the stack deco…
We consider the problem of constructing codes that can correct $δ$ deletions occurring in an arbitrary binary string of length $n$ bits. Varshamov-Tenengolts (VT) codes can correct all possible single deletions $(δ=1)$ with an asymptotically optimal redundancy. Finding similar codes for $δ\geq 2$ deletions is an open problem. We propose a new family of codes, that we call Guess & Check (GC) codes, that can correct, with high probability, a constant number of deletions $δ$ occurring at uniformly random positions within an arbitrary string. The GC codes are based on MDS codes and have an asympto…
In this paper we present an optimized parallel implementation of a flexible MAP decoder for synchronization error correcting codes, supporting a very wide range of code sizes and channel conditions. On mid-range GPUs we demonstrate decoding speedups of more than two orders of magnitude over a CPU implementation of the same optimized algorithm, and more than an order of magnitude over our earlier GPU implementation. The prominent challenge is to maintain high parallelization efficiency over a wide range of code sizes and channel conditions, and different execution hardware. We ensure this with …
Since a quantum measurement generally disturbs the state of a quantum system, one might think that it should not be possible for a sender and receiver to communicate reliably when the receiver performs a large number of sequential measurements to determine the message of the sender. We show here that this intuition is not true, by demonstrating that a sequential decoding strategy works well even in the most general "one-shot" regime, where we are given a single instance of a channel and wish to determine the maximal number of bits that can be communicated up to a small failure probability. Thi…
In this paper, we study how often unique decoding from $t$ insertions or $t$ deletions occurs for error correcting codes. Insertions and deletions frequently occur in synchronization problems and DNA, a medium which is beginning to be used for long term data storage. We define natural probabilistic channels that make $t$ insertions or $t$ deletions, and study the probability of unique decoding. Our most substantial contribution is the derivation of tight upper bounds on the probability of unique decoding for messages passed though these channels. We also consider other aspects of the problem, …
Considering a short frame length, which is typical in Ultra-Reliable Low-Latency and massive Machine Type Communications, a trade-off exists between improving the performance of frame synchronization (FS) and improving the performance of information throughput. In this paper, we consider the case of continuous transmission over AWGN channels where the synchronization sequence is superimposed to the data symbols, as opposed to being added as a frame header. The advantage of this superposition is that the synchronization length is as long as the frame length. On the other hand, its power has to …
In this paper, we present an optimal metric function on average, which leads to a significantly low decoding computation while maintaining the superiority of the polarization-adjusted convolutional (PAC) codes' error-correction performance. With our proposed metric function, the PAC codes' decoding computation is comparable to the conventional convolutional codes (CC) sequential decoding. Moreover, simulation results show an improvement in the low-rate PAC codes' error-correction performance when using our proposed metric function. We prove that choosing the polarized cutoff rate as the metric…
In \cite{Chandar2008}, Chandar et al studied a problem of sequential frame synchronization for a frame transmitted randomly and uniformly among $A$ slots. For a discrete memory-less channel (DMC), they showed that the frame length $N$ must scale as $e^{α(Q)N}>A$ for the frame detection error to go to zero asymptotically with $A$. $α(Q)$ is the synchronization threshold and $Q$ is channel transition probability. We study the sequential frame synchronisation problem for a fading channel and additive noise and seek to characterise the effect of fading. For a discrete ON-OFF fading channel (with O…
Low-Density Parity-Check (LDPC) codes are usually decoded by running an iterative belief-propagation, or message-passing, algorithm over the factor graph of the code. The traditional message-passing schedule consists of updating all the variable nodes in the graph, using the same pre-update information, followed by updating all the check nodes of the graph, again, using the same pre-update information. Recently several studies show that sequential scheduling, in which messages are generated using the latest available information, significantly improves the convergence speed in terms of number …
In this paper, we are interested in the performance of a variable-length stop-feedback (VLSF) code with $m$ optimal decoding times for the binary-input additive white Gaussian noise channel. We first develop tight approximations on the tail probability of length-$n$ cumulative information density. Building on the work of Yavas \emph{et al.}, for a given information density threshold, we formulate the integer program of minimizing the upper bound on average blocklength over all decoding times subject to the average error probability, minimum gap and integer constraints. Eventually, minimization…
Polar codes are able to achieve the capacity of memoryless channels under successive cancellation (SC) decoding. Soft Cancellation (SCAN) is a soft-output decoder based on the SC schedule, useful in iterative decoding and concatenation of polar codes. However, the sequential nature of this decoder leads to high decoding latency compared to state-of-the-art codes. To reduce the latency of SCAN, in this paper we identify special nodes in the decoding tree, corresponding to specific frozen-bit sequences, and propose dedicated low-latency decoding approaches for each of them. The resulting fast-SC…
In this paper we consider an interacting two-agent sequential decision-making problem consisting of a Markov source process, a causal encoder with feedback, and a causal decoder. Motivated by a desire to foster links between control and information theory, we augment the standard formulation by considering general alphabets and a cost function operating on current and previous symbols. Using dynamic programming, we provide a structural result whereby an optimal scheme exists that operates on appropriate sufficient statistics. We emphasize an example where the decoder alphabet lies in a space o…
A complexity-adaptive tree search algorithm is proposed for $\boldsymbol{G}_N$-coset codes that implements maximum-likelihood (ML) decoding by using a successive decoding schedule. The average complexity is close to that of the successive cancellation (SC) decoding for practical error rates when applied to polar codes and short Reed-Muller (RM) codes, e.g., block lengths up to $N=128$. By modifying the algorithm to limit the worst-case complexity, one obtains a near-ML decoder for longer RM codes and their subcodes. Unlike other bit-flip decoders, no outer code is needed to terminate decoding.…
\begin{abstract} In this paper we consider Time-Varying Block (TVB) codes, which generalize a number of previous synchronization error-correcting codes. We also consider various practical issues related to MAP decoding of these codes. Specifically, we give an expression for the expected distribution of drift between transmitter and receiver due to synchronization errors. We determine an appropriate choice for state space limits based on the drift probability distribution. In turn, we obtain an expression for the decoder complexity under given channel conditions in terms of the state space limi…
A $K$-user memoryless interference channel is considered where each receiver sequentially decodes the data of a subset of transmitters before it decodes the data of the designated transmitter. Therefore, the data rate of each transmitter depends on (i) the subset of receivers which decode the data of that transmitter, (ii) the decoding order, employed at each of these receivers. In this paper, a greedy algorithm is developed to find the users which are decoded at each receiver and the corresponding decoding order such that the minimum rate of the users is maximized. It is proven that the propo…
Though the concept of non-orthogonal multiple access (NOMA) was proposed several years ago, the performance of uplink NOMA has only been verified in theory, but not in practice. This paper presents an over-the-air implementation of a uplink NOMA system, while providing solutions to most common practical problems, i.e., carrier frequency offset (CFO) synchronization, time synchronization, and channel estimation. The implemented CFO synchronization method adopts the primary synchronization signal (PSS) of LTE. Also, we design a novel preamble for each uplink user, and it is appended to every fra…
Polar coding was conceived originally as a technique for boosting the cutoff rate of sequential decoding, along the lines of earlier schemes of Pinsker and Massey. The key idea in boosting the cutoff rate is to take a vector channel (either given or artificially built), split it into multiple correlated subchannels, and employ a separate sequential decoder on each subchannel. Polar coding was originally designed to be a low-complexity recursive channel combining and splitting operation of this type, to be used as the inner code in a concatenated scheme with outer convolutional coding and seque…
This paper investigates the problem of joint phase tracking and channel decoding in OFDM based Physical-layer Network Coding (PNC) systems. OFDM signaling can obviate the need for tight time synchronization among multiple simultaneous transmissions in the uplink of PNC systems. However, OFDM PNC systems are susceptible to phase drifts caused by residual carrier frequency offsets (CFOs). In the traditional OFDM system in which a receiver receives from only one transmitter, pilot tones are employed to aid phase tracking. In OFDM PNC systems, multiple transmitters transmit to a receiver, and thes…
Owing to their capacity-achieving performance and low encoding and decoding complexity, polar codes have drawn much research interests recently. Successive cancellation decoding (SCD) and belief propagation decoding (BPD) are two common approaches for decoding polar codes. SCD is sequential in nature while BPD can run in parallel. Thus BPD is more attractive for low latency applications. However BPD has some performance degradation at higher SNR when compared with SCD. Concatenating LDPC with Polar codes is one popular approach to enhance the performance of BPD , where a short LDPC code is use…
We study a problem of sequential frame synchronization for a frame transmitted uniformly in $A$ slots. For a discrete memoryless channel (DMC), Venkat Chandar et al showed that the frame length $N$ must scale with $A$ as $e^{N α(Q)} > A$ for the frame synchronization error to go to zero (asymptotically with $A$). Here, $Q$ denotes the transition probabilities of the DMC and $α(Q)$, defined as the synchronization threshold, characterizes the scaling needed of $N$ for asymptotic error free frame synchronization. We show that the asynchronous communication framework permits a natural tradeoff bet…
