Turbo coding is a powerful class of forward error correcting codes, which can achieve performances close to the Shannon limit. The turbo principle can be applied to the problem of side-information source coding, and we investigate here its application to the reconciliation problem occurring in a continuous-variable quantum key distribution protocol.
In source coding, either with or without side information at the decoder, the ultimate performance can be achieved by means of random binning. Structured binning into cosets of performing channel codes has been successfully employed in practical applications. In this letter it is formally shown that various convolutional- and turbo-syndrome decoding algorithms proposed in literature lead in fact to the same estimate. An equivalent implementation is also delineated by directly tackling syndrome decoding as a maximum a posteriori probability problem and solving it by means of iterative message-p…
We consider compressive sensing as a source coding method for signal transmission. We concatenate a convolutional coding system with 1-bit compressive sensing to obtain a serial concatenated system model for sparse signal transmission over an AWGN channel. The proposed source/channel decoder, which we refer to as turbo CS, is robust against channel noise and its signal reconstruction performance at the receiver increases considerably through iterations. We show 12 dB improvement with six turbo CS iterations compared to a non-iterative concatenated source/channel decoder.
Quantum stabilizer codes (QSCs) suffer from a low quantum coding rate, since they have to recover the quantum bits (qubits) in the face of both bit-flip and phase-flip errors. In this treatise, we conceive a low-complexity concatenated quantum turbo code (QTC) design exhibiting a high quantum coding rate. The high quantum coding rate is achieved by combining the quantum-domain version of short-block codes (SBCs) also known as single parity check (SPC) codes as the outer codes and quantum unity-rate codes (QURCs) as the inner codes. Despite its design simplicity, the proposed QTC yields a near-…
Conventional turbo codes (CTCs) usually employ a block-oriented interleaving so that each block is separately encoded and decoded. As interleaving and de-interleaving are performed within a block, the message-passing process associated with an iterative decoder is limited to proceed within the corresponding range. This paper presents a new turbo coding scheme that uses a special interleaver structure and a multiple-round early termination test involving both sign check and a CRC code. The new interleaver structure is naturally suited for high speed parallel processing and the resulting coding …
The use of active narrow-band interference (NBI) suppression in direct-sequence spread-spectrum (DS-SS) communications has been extensively studied. In this paper, we address the problem of optimum coding-spreading tradeoff for NBI suppression. With maximum likelihood decoding, we first derive upper bounds on the error probability of coded systems in the presence of a special class of NBI, namely, multi-tone interference with orthogonal signatures. By employing the well-developed bounding techniques, we show there is no advantage in spreading, and hence a low-rate full coding approach is alway…
In this paper, the design of irregular turbo codes for the binary erasure channel is investigated. An analytic expression of the erasure probability of punctured recursive systematic convolutional codes is derived. This exact expression will be used to track the density evolution of turbo codes over the erasure channel, that will allow for the design of capacity-approaching irregular turbo codes. Next, we propose a graph-optimal interleaver for irregular turbo codes. Simulation results for different coding rates is shown at the end.
This paper proposes a novel deep learning-based error correction coding scheme for AWGN channels under the constraint of one-bit quantization in the receivers. Specifically, it is first shown that the optimum error correction code that minimizes the probability of bit error can be obtained by perfectly training a special autoencoder, in which "perfectly" refers to converging the global minima. However, perfect training is not possible in most cases. To approach the performance of a perfectly trained autoencoder with a suboptimum training, we propose utilizing turbo codes as an implicit regular…
In this paper, iterative differential phase-shift keying (DPSK) demodulation and channel decoding scheme is investigated for the Joint Channel decoding and physical layer Network Coding (JCNC) approach in two-way relaying systems. The Bahl, Cocke, Jelinek, and Raviv (BCJR) algorithm for both coherent and noncoherent detection is derived for soft-in soft-out decoding of DPSK signalling over the two-user multiple-access channel with Rayleigh fading. Then, we propose a pragmatic approach with the JCNC scheme for iteratively exploiting the extrinsic information of the outer code. With coherent det…
In this paper we present a new Turbo analog error correcting coding scheme for real valued signals that are corrupted by impulsive noise. This Turbo code improves Donoho's deterministic construction by using a probabilistic approach. More specifically, our construction corrects more errors than the matrices of Donoho by allowing a vanishingly small probability of error (with the increase in block size). The problem of decoding the long block code is decoupled into two sets of parallel Linear Programming problems. This leads to a significant reduction in decoding complexity as compared to one-s…
In this paper we investigate the decoding of parallel turbo codes over the binary erasure channel suited for upper-layer error correction. The proposed algorithm performs on-the-fly decoding, i.e. it starts decoding as soon as the first symbols are received. This algorithm compares with the iterative decoding of codes defined on graphs, in that it propagates in the trellises of the turbo code by removing transitions in the same way edges are removed in a bipartite graph under message-passing decoding. Performance comparison with LDPC codes for different coding rates is shown.
Guessing Random Additive Noise Decoding (GRAND) is a family of universal decoding algorithms suitable for decoding any moderate redundancy code of any length. We establish that, through the use of list decoding, soft-input variants of GRAND can replace the Chase algorithm as the component decoder in the turbo decoding of product codes. In addition to being able to decode arbitrary product codes, rather than just those with dedicated hard-input component code decoders, results show that ORBGRAND achieves a coding gain of up to 0.7dB over the Chase algorithm with same list size.
As wireless communication systems look intently to compose the transition from voice communication to interactive Internet data, achieving higher bit rates becomes both increasingly desirable and challenging. Space-time coding (STC) is a communications technique for wireless systems that inhabit multiple transmit antennas and single or multiple receive antennas. Space-time codes make use of advantage of both the spatial diversity provided by multiple antennas and the temporal diversity available with time-varying fading. Space-time codes can be divided into block codes and trellis codes. Space…
Certain binary asymmetric channels, such as Z-channels in which one of the two crossover probabilities is zero, demand optimal ones densities different from 50%. Some broadcast channels, such as broadcast binary symmetric channels (BBSC) where each component channel is a binary symmetric channel, also require a non-uniform input distribution due to the superposition coding scheme, which is known to achieve the boundary of capacity region. This paper presents a systematic technique for designing nonlinear turbo codes that are able to support ones densities different from 50%. To demonstrate the…
In this paper a new class of lattices called turbo lattices is introduced and established. We use the lattice Construction D to produce turbo lattices. This method needs a set of nested linear codes as its underlying structure. We benefit from turbo codes as our basis codes. Therefore, a set of nested turbo codes based on nested interleavers (block interleavers) and nested convolutional codes is built. To this end, we employ both tail-biting and zero-tail convolutional codes. Using these codes, along with construction D, turbo lattices are created. Several properties of Construction D lattices…
A critical aspect of reliable communication involves the design of codes that allow transmissions to be robustly and computationally efficiently decoded under noisy conditions. Advances in the design of reliable codes have been driven by coding theory and have been sporadic. Recently, it is shown that channel codes that are comparable to modern codes can be learned solely via deep learning. In particular, Turbo Autoencoder (TURBOAE), introduced by Jiang et al., is shown to achieve the reliability of Turbo codes for Additive White Gaussian Noise channels. In this paper, we focus on applying the…
Attracted by its scalability towards practical codeword lengths, we revisit the idea of Turbo-autoencoders for end-to-end learning of PHY-Layer communications. For this, we study the existing concepts of Turbo-autoencoders from the literature and compare the concept with state-of-the-art classical coding schemes. We propose a new component-wise training algorithm based on the idea of Gaussian a priori distributions that reduces the overall training time by almost a magnitude. Further, we propose a new serial architecture inspired by classical serially concatenated Turbo code structures and sho…
The purpose of this study is to construct a near capacity Irregular Turbo Code and to evaluate its performance over Gaussian channel. The methodology used to evaluate and measure the performance of the new design is by simulating the system by developing a software platform using Matlab. The simulation carried out by implementing the system over different modulation schemes, different frame sizes, and different code rates in order to achieve a fair comparison between irregular turbo code and regular Turbo Code. The simulation results showed that the Irregular Turbo Code with 64QAM modulation i…
An unexpected breakdown in the existing theory of quantum serial turbo coding is that a quantum convolutional encoder cannot simultaneously be recursive and non-catastrophic. These properties are essential for quantum turbo code families to have a minimum distance growing with blocklength and for their iterative decoding algorithm to converge, respectively. Here, we show that the entanglement-assisted paradigm simplifies the theory of quantum turbo codes, in the sense that an entanglement-assisted quantum (EAQ) convolutional encoder can possess both of the aforementioned desirable properties. …
The aim of this paper is to study the performance of some coding schemes recently proposed for updating the TC channel coding standard for space applications, in the presence of jamming. Besides low-density parity-check codes, that appear as the most eligible candidates, we also consider other solutions based on parallel turbo codes and extended BCH codes. We show that all these schemes offer very good performance, which approaches the theoretical limits achievable.
University of Washington(华盛顿大学);Samsung AI Center(三星人工智能中心);Tata Institute of Fundamental Research(塔塔基础研究院);University of Illinois at Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校)
Designing codes that combat the noise in a communication medium has remained a significant area of research in information theory as well as wireless communications. Asymptotically optimal channel codes have been developed by mathematicians for communicating under canonical models after over 60 years of research. On the other hand, in many non-canonical channel settings, optimal codes do not exist and the codes designed for canonical models are adapted via heuristics to these channels and are thus not guaranteed to be optimal. In this work, we make significant progress on this problem by desig…
We consider the problem of compressing memoryless binary data with or without side information at the decoder. We review the parity- and the syndrome-based approaches and discuss their theoretical limits, assuming that there exists a virtual binary symmetric channel between the source and the side information, and that the source is not necessarily uniformly distributed. We take a factor-graph-based approach in order to devise how to take full advantage of the ready-available iterative decoding procedures when turbo codes are employed, in both a parity- or a syndrome-based fashion. We end up o…
An interleaver is a critical component for the channel coding performance of turbo codes. Algebraic constructions are of particular interest because they admit analytical designs and simple, practical hardware implementation. Contention-free interleavers have been recently shown to be suitable for parallel decoding of turbo codes. In this correspondence, it is shown that permutation polynomials generate maximum contention-free interleavers, i.e., every factor of the interleaver length becomes a possible degree of parallel processing of the decoder. Further, it is shown by computer simulations …
In this paper, we establish an interesting duality between two different quantum information-processing tasks, namely, classical source coding with quantum side information, and channel coding over c-q channels. The duality relates the optimal error exponents of these two tasks, generalizing the classical results of Ahlswede and Dueck. We establish duality both at the operational level and at the level of the entropic quantities characterizing these exponents. For the latter, the duality is given by an exact relation, whereas for the former, duality manifests itself in the following sense: an …
In this paper, a distributed turbo-like coding scheme for wireless networks with relays is proposed. We consider a scenario where multiple sources communicate with a single destination with the help of a relay. The proposed scheme can be regarded as of the decode-and-forward type. The relay decodes the information from the sources and it properly combines and re-encodes them to generate some extra redundancy, which is transmitted to the destination. The amount of redundancy generated by the relay can simply be adjusted according to requirements in terms of performance, throughput and/or power.…
An interleaver is a critical component for the channel coding performance of turbo codes. Algebraic constructions are important because they admit analytical designs and simple, practical hardware implementation. The spread factor of an interleaver is a common measure for turbo coding applications. Maximum-spread interleavers are interleavers whose spread factors achieve the upper bound. An infinite sequence of quadratic permutation polynomials over integer rings that generate maximum-spread interleavers is presented. New properties of permutation polynomial interleavers are investigated from …
We consider the distributed source coding problem in which correlated data picked up by scattered sensors has to be encoded separately and transmitted to a common receiver, subject to a rate-distortion constraint. Although near-tooptimal solutions based on Turbo and LDPC codes exist for this problem, in most cases the proposed techniques do not scale to networks of hundreds of sensors. We present a scalable solution based on the following key elements: (a) distortion-optimized index assignments for low-complexity distributed quantization, (b) source-optimized hierarchical clustering based on t…
This paper studies variable-length (VL) source coding of general sources with side-information. Novel one-shot coding theorems for coding with common side-information available at the encoder and the decoder and Slepian- Wolf (SW) coding (i.e., with side-information only at the decoder) are given, and then, are applied to asymptotic analyses of these coding problems. Especially, a general formula for the infimum of the coding rate asymptotically achievable by weak VL-SW coding (i.e., VL-SW coding with vanishing error probability) is derived. Further, the general formula is applied to investiga…
The problem of source coding with side information (SCSI) is closely related to channel coding. Therefore, existing literature focuses on using the most successful channel codes namely, LDPC codes, turbo codes, and their variants, to solve this problem assuming classical unique decoding of the underlying channel code. In this paper, in contrast to classical decoding, we have taken the list decoding approach. We show that syndrome source coding using list decoding can achieve the theoretical limit. We argue that, as opposed to channel coding, the correct sequence from the list produced by the l…
We consider source coding with fixed lag side information at the decoder. We focus on the special case of perfect side information with unit lag corresponding to source coding with feedforward (the dual of channel coding with feedback) introduced by Pradhan. We use this duality to develop a linear complexity algorithm which achieves the rate-distortion bound for any memoryless finite alphabet source and distortion measure.
This paper studies a large random matrix system (LRMS) model involving an arbitrary signal distribution and forward error control (FEC) coding. We establish an area property based on the so-called Turbo approximate message passing (Turbo-AMP) algorithm. Under the assumption that the state evolution for AMP is correct for the coded system, the achievable rate of Turbo-AMP is analyzed. We prove that Turbo-AMP achieves the constraint capacity of the LRMS with an arbitrary signal distribution provided that a matching condition is satisfied. As a byproduct, we provide an alternative derivation for …
Distributed source coding schemes are typically based on the use of channels codes as source codes. In this paper we propose a new paradigm, termed "distributed arithmetic coding", which exploits the fact that arithmetic codes are good source as well as channel codes. In particular, we propose a distributed binary arithmetic coder for Slepian-Wolf coding with decoder side information, along with a soft joint decoder. The proposed scheme provides several advantages over existing Slepian-Wolf coders, especially its good performance at small block lengths, and the ability to incorporate arbitrary…
The error-pattern correcting code (EPCC) is incorporated in the design of a turbo equalizer (TE) with aim to correct dominant error events of the inter-symbol interference (ISI) channel at the output of its matching Viterbi detector. By targeting the low Hamming-weight interleaved errors of the outer convolutional code, which are responsible for low Euclidean-weight errors in the Viterbi trellis, the turbo equalizer with an error-pattern correcting code (TE-EPCC) exhibits a much lower bit-error rate (BER) floor compared to the conventional non-precoded TE, especially for high rate applications…
We investigate and compare different options for updating the error correcting code currently used in space mission telecommand links. Taking as a reference the solutions recently emerged as the most promising ones, based on Low-Density Parity-Check codes, we explore the behavior of alternative schemes, based on parallel concatenated turbo codes and soft-decision decoded BCH codes. Our analysis shows that these further options can offer similar or even better performance.
This letter investigates the role of index coding in the capacity of AWGN broadcast channels with receiver message side information. We first show that index coding is unnecessary where there are two receivers; multiplexing coding and superposition coding are sufficient to achieve the capacity region. We next show that, for more than two receivers, multiplexing coding and superposition coding alone can be suboptimal. We give an example where these two coding schemes alone cannot achieve the capacity region, but adding index coding can. This demonstrates that, in contrast to the two-receiver ca…
Security aspects of the Index Coding with Side Information (ICSI) problem are investigated. Building on the results of Bar-Yossef et al. (2006), the properties of linear coding schemes for the ICSI problem are further explored. The notion of weak security, considered by Bhattad and Narayanan (2005) in the context of network coding, is generalized to block security. It is shown that the coding scheme for the ICSI problem based on a linear code C of length n, minimum distance d and dual distance d^\perp, is (d-1-t)-block secure (and hence also weakly secure) if the adversary knows in advance t \…
In this paper, new index coding problems are studied, where each receiver has erroneous side information. Although side information is a crucial part of index coding, the existence of erroneous side information has not yet been considered. We study an index code with receivers that have erroneous side information symbols in the error-free broadcast channel, which is called an index code with side information errors (ICSIE). The encoding and decoding procedures of the ICSIE are proposed, based on the syndrome decoding. Then, we derive the bounds on the optimal codelength of the proposed index c…
Let (S1,i, S2,i), distributed according to i.i.d p(s1, s2), i = 1, 2, . . . be a memoryless, correlated partial side information sequence. In this work we study channel coding and source coding problems where the partial side information (S1, S2) is available at the encoder and the decoder, respectively, and, additionally, either the encoder's or the decoder's side information is increased by a limited-rate description of the other's partial side information. We derive six special cases of channel coding and source coding problems and we characterize the capacity and the rate-distortion functi…
This paper considers the problem of source coding with side information at the decoder, also called Slepian-Wolf source coding scheme. In practical applications of this coding scheme, the statistical relation between the source and the side information can vary from one data transmission to another, and there is a need to adapt the coding rate depending on the current statistical relation. In this paper, we propose a novel rate-adaptive code construction based on LDPC codes for the Slepian-Wolf source coding scheme. The proposed code design method allows to optimize the code degree distributio…
This paper simplifies an existing coding scheme for the two-receiver discrete memoryless broadcast channel with complementary receiver side information where there is a passive eavesdropper and individual secrecy is required. The existing coding scheme is simplified in two steps by replacing Wyner secrecy coding with Carleial-Hellman secrecy coding. The resulting simplified scheme is free from redundant message splits and random components. Not least, the simplified scheme retains the existing achievable individual secrecy rate region. Finally, its construction simplicity helps us gain additio…
This paper introduces a new maximum likelihood (ML) solution for the code-aided (CA) timing recovery problem in square-QAM transmissions and derives, for the very first time, its CA Cramér-Rao lower bounds (CRLBs) in closed-form expressions. By exploiting the full symmetry of square-QAM constellations and further scrutinizing the Gray-coding mechanism, we express the likelihood function (LF) of the system explicitly in terms of the code bits' \textit{a priori} log-likelihood ratios (LLRs). The timing recovery task is then embedded in the turbo iteration loop wherein increasingly accurate estim…
A multiterminal lossy coding problem, which includes various problems such as the Wyner-Ziv problem and the complementary delivery problem as special cases, is considered. It is shown that any point in the achievable rate-distortion region can be attained even if the source statistics are not known.
It is known that the frame error rate of turbo codes on quasi-static fading channels can be accurately approximated using the convergence threshold of the corresponding iterative decoder. This paper considers quasi-static fading channels and demonstrates that non-iterative schemes can also be characterized by a similar threshold based on which their frame error rate can be readily estimated. In particular, we show that this threshold is a function of the probability of successful frame detection in additive white Gaussian noise, normalized by the squared instantaneous signal-to-noise ratio. We…
A lossy source coding problem is studied in which a source encoder communicates with two decoders, one with and one without correlated side information with an additional constraint on the privacy of the side information at the uninformed decoder. Two cases of this problem arise depending on the availability of the side information at the encoder. The set of all feasible rate-distortion-equivocation tuples are characterized for both cases. The difference between the informed and uninformed cases and the advantages of encoder side information for enhancing privacy are highlighted for a binary s…
In this paper, we study the upper and the lower bounds on the joint source-channel coding error exponent with decoder side-information. The results in the paper are non-trivial extensions of the Csiszar's classical paper [5]. Unlike the joint source-channel coding result in [5], it is not obvious whether the lower bound and the upper bound are equivalent even if the channel coding error exponent is known. For a class of channels, including the symmetric channels, we apply a game-theoretic result to establish the existence of a saddle point and hence prove that the lower and upper bounds are th…
One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Renyi entropy and smooth max Renyi divergence. Asymptotically (in the limit of large block lengths) this region is quantified in terms of spectral-sup conditional entropy rate and spectral- sup mutual information rate. In particular, it coincides with the rate region derived in the limit of unlimited independent and identically distributed copies of the sources.
We study the hybrid classical-quantum version of the channel coding problem for the famous Gel'fand-Pinsker channel. In the classical setting for this channel the conditional distribution of the channel output given the channel input is a function of a random parameter called the channel state. We study this problem when a rate limited version of the channel state is available at the encoder for the classical-quantum Gel'fand-Pinsker channel. We establish the capacity region for this problem in the information-spectrum setting. The capacity region is quantified in terms of spectral-sup classic…
The model of a side information "vending machine" accounts for scenarios in which acquiring side information is costly and thus should be done efficiently. In this paper, the three-node cascade source coding problem is studied under the assumption that a side information vending machine is available either at the intermediate or at the end node. In both cases, a single-letter characterization of the available trade-offs among the rate, the distortions in the reconstructions at the intermediate and at the end node, and the cost in acquiring the side information are derived under given condition…
We study source coding in the presence of side information, when the system can take actions that affect the availability, quality, or nature of the side information. We begin by extending the Wyner-Ziv problem of source coding with decoder side information to the case where the decoder is allowed to choose actions affecting the side information. We then consider the setting where actions are taken by the encoder, based on its observation of the source. Actions may have costs that are commensurate with the quality of the side information they yield, and an overall per-symbol cost constraint ma…
We study error exponents for source coding with side information. Both achievable exponents and converse bounds are obtained for the following two cases: lossless source coding with coded information (SCCSI) and lossy source coding with full side information (Wyner-Ziv). These results recover and extend several existing results on source-coding error exponents and are tight in some circumstances. Our bounds have a natural interpretation as a two-player game between nature and the code designer, with nature seeking to minimize the exponent and the code designer seeking to maximize it. In the Wy…
Consider the problem of source coding in networks with multiple receiving terminals, each having access to some kind of side information. In this case, standard coding techniques are either prohibitively complex to decode, or require network-source coding separation, resulting in sub-optimal transmission rates. To alleviate this problem, we offer a joint network-source coding scheme based on matrix sparsification at the code design phase, which allows the terminals to use an efficient decoding procedure (syndrome decoding using LDPC), despite the network coding throughout the network. Via a no…
We consider multi-terminal source coding with a single encoder and multiple decoders where either the encoder or the decoders can take cost constrained actions which affect the quality of the side information present at the decoders. For the scenario where decoders take actions, we characterize the rate-cost trade-off region for lossless source coding, and give an achievability scheme for lossy source coding for two decoders which is optimum for a variety of special cases of interest. For the case where the encoder takes actions, we characterize the rate-cost trade-off for a class of lossless …
The problem of lossless fixed-rate streaming coding of discrete memoryless sources with side information at the decoder is studied. A random time-varying tree-code is used to sequentially bin strings and a Stack Algorithm with a variable bias uses the side information to give a delay-universal coding system for lossless source coding with side information. The scheme is shown to give exponentially decaying probability of error with delay, with exponent equal to Gallager's random coding exponent for sources with side information. The mean of the random variable of computation for the stack deco…
We consider the cascade and triangular rate-distortion problem where side information is known to the source encoder and to the first user but not to the second user. We characterize the rate-distortion region for these problems. For the quadratic Gaussian case, we show that it is sufficient to consider jointly Gaussian distributions, a fact that leads to an explicit solution.
An interleaver is a critical component for the channel coding performance of turbo codes. Algebraic constructions are of particular interest because they admit analytical designs and simple, practical hardware implementation. Also, the recently proposed quadratic permutation polynomial (QPP) based interleavers by Sun and Takeshita (IEEE Trans. Inf. Theory, Jan. 2005) provide excellent performance for short-to-medium block lengths, and have been selected for the 3GPP LTE standard. In this work, we derive some upper bounds on the best achievable minimum distance dmin of QPP-based conventional bi…
The output of a discrete Markov source is to be encoded instantaneously by a variable-rate encoder and decoded by a finite-state decoder. Our performance measure is a linear combination of the distortion and the instantaneous rate. Structure theorems, pertaining to the encoder and next-state functions are derived for every given finite-state decoder, which can have access to side information.
An interleaver is a critical component for the channel coding performance of turbo codes. Algebraic constructions are of particular interest because they admit analytical designs and simple, practical hardware implementation. Sun and Takeshita have recently shown that the class of quadratic permutation polynomials over integer rings provides excellent performance for turbo codes. In this correspondence, a necessary and sufficient condition is proven for the existence of a quadratic inverse polynomial for a quadratic permutation polynomial over an integer ring. Further, a simple construction is…