We recently proved threshold saturation for spatially coupled sparse superposition codes on the additive white Gaussian noise channel. Here we generalize our analysis to a much broader setting. We show for any memoryless channel that spatial coupling allows generalized approximate message-passing (GAMP) decoding to reach the potential (or Bayes optimal) threshold of the code ensemble. Moreover in the large input alphabet size limit: i) the GAMP algorithmic threshold of the underlying (or uncoupled) code ensemble is simply expressed as a Fisher information; ii) the potential threshold tends to Shannon's capacity. Although we focus on coding for sake of coherence with our previous results, the framework and methods are very general and hold for a wide class of generalized estimation problems with random linear mixing.
We recently showed in [1] the superiority of certain structured coding matrices ensembles (such as partial row-orthogonal) for sparse superposition codes when compared with purely random matrices with i.i.d. entries, both information-theoretically and under practical vector approximate message-passing decoding. Here we generalize this result to binary input channels under generalized vector approximate message-passing decoding [2].We focus on specific binary output channels for concreteness but our analysis based on the replica symmetric method from statistical physics applies to any memoryles…
We study a class of discrete memoryless broadcast interference channels (DM-BICs), where one of the broadcast receivers is subject to the interference from a point-to-point transmission. A general achievable rate region $\mathcal{R}$ based on rate splitting, superposition coding and binning at the broadcast transmitter and rate splitting at the interfering transmitter is derived. Under two partial order broadcast conditions {\em interference-oblivious less noisy} and {\em interference-cognizant less noisy}, a reduced form of $\mathcal{R}$ is shown to be equivalent to the region based on a simp…
We consider spatially coupled code ensembles. A particular instance are convolutional LDPC ensembles. It was recently shown that, for transmission over the memoryless binary erasure channel, this coupling increases the belief propagation threshold of the ensemble to the maximum a-posteriori threshold of the underlying component ensemble. This paved the way for a new class of capacity achieving low-density parity check codes. It was also shown empirically that the same threshold saturation occurs when we consider transmission over general binary input memoryless channels. In this work, we repor…
In this paper, we study the state-dependent two-user interference channel, where the state information is non-causally known at both transmitters but unknown to either of the receivers. We propose two coding schemes for the discrete memoryless case: simultaneous encoding for the sub-messages in the first one and superposition encoding in the second one, both with rate splitting and Gel'fand-Pinsker coding. The corresponding achievable rate regions are established.
Recently, a new class of codes, called sparse superposition or sparse regression codes, has been proposed for communication over the AWGN channel. It has been proven that they achieve capacity using power allocation and various forms of iterative decoding. Empirical evidence has also strongly suggested that the codes achieve capacity when spatial coupling and approximate message passing decoding are used, without need of power allocation. In this note we prove that state evolution (which tracks message passing) indeed saturates the potential threshold of the underlying code ensemble, which app…
This paper studies the mismatched decoding problem for binary-input discrete memoryless channels. An example is provided for which an achievable rate based on superposition coding exceeds the LM rate (Hui, 1983; Csiszár-Körner, 1981), thus providing a counter-example to a previously reported converse result (Balakirsky, 1995). Both numerical evaluations and theoretical results are used in establishing this claim.
Sparse superposition codes, or sparse regression codes, constitute a new class of codes which was first introduced for communication over the additive white Gaussian noise (AWGN) channel. It has been shown that such codes are capacity-achieving over the AWGN channel under optimal maximum-likelihood decoding as well as under various efficient iterative decoding schemes equipped with power allocation or spatially coupled constructions. Here, we generalize the analysis of these codes to a much broader setting that includes all memoryless channels. We show, for a large class of memoryless channels…
In this paper, we study the finite blocklength limits of state-dependent discrete memoryless channels where the discrete memoryless state is known noncausally at the encoder. For the point-to-point case, this is known as the Gel'fand-Pinsker channel model. We define the (n,ε)-capacity of the Gel'fand-Pinsker channel as the maximal rate of transmission of a message subject to the condition that the length of the block-code is n and the average error probability is no larger than ε. This paper provides a lower bound for the (n,ε)-capacity of the Gel'fand-Pinsker channel model, and hence an upper…
Bounds on the reliability function for the discrete memoryless relay channel are derived using the method of types. Two achievable error exponents are derived based on partial decode-forward and compress-forward which are well-known superposition block-Markov coding schemes. The derivations require combinations of the techniques involved in the proofs of Csiszár-Körner-Marton's packing lemma for the error exponent of channel coding and Marton's type covering lemma for the error exponent of source coding with a fidelity criterion. The decode-forward error exponent is evaluated on Sato's relay c…
Previous works established the capacity region for some special cases of discrete memoryless degraded cognitive interference channel (CIC) with unidirectional destination cooperation (UDC). In this letter, we characterize the capacity region of the general discrete memoryless degraded CIC-UDC. The obtained results imply that the capacity region is achieved by the Gel'fand-Pinsker coding at the cognitive transmitter, superposition coding at the primary transmitter and decode-and-forward at the relay. Furthermore, using this general result and a novel converse analysis, we establish the capacity…
We consider spatially coupled code ensembles. A particular instance are convolutional LDPC ensembles. It was recently shown that, for transmission over the binary erasure channel, this coupling increases the belief propagation threshold of the ensemble to the maximum a-priori threshold of the underlying component ensemble. We report on empirical evidence which suggest that the same phenomenon also occurs when transmission takes place over a general binary memoryless symmetric channel. This is confirmed both by simulations as well as by computing EBP GEXIT curves and by comparing the empirical …
A new single-letter achievable rate region is proposed for the two-user discrete memoryless multiple-access channel(MAC) with noiseless feedback. The proposed region includes the Cover-Leung rate region [1], and it is shown that the inclusion is strict. The proof uses a block-Markov superposition strategy based on the observation that the messages of the two users are correlated given the feedback. The rates of transmission are too high for each encoder to decode the other's message directly using the feedback, so they transmit correlated information in the next block to learn the message of o…
Convolutional low-density parity-check (LDPC) codes (or spatially-coupled codes) have now been shown to achieve capacity on binary-input memoryless symmetric channels. The principle behind this surprising result is the threshold-saturation phenomenon, which is defined by the belief-propagation threshold of the spatially-coupled ensemble saturating to a fundamental threshold defined by the uncoupled system. Previously, the authors demonstrated that potential functions can be used to provide a simple proof of threshold saturation for coupled scalar recursions. In this paper, we present a simple …
In this paper, we propose an achievable rate region for discrete memoryless interference channels with conferencing at the transmitter side. We employ superposition block Markov encoding, combined with simultaneous superposition coding, dirty paper coding, and random binning to obtain the achievable rate region. We show that, under respective conditions, the proposed achievable region reduces to Han and Kobayashi achievable region for interference channels, the capacity region for degraded relay channels, and the capacity region for the Gaussian vector broadcast channel. Numerical examples for…
In this paper we are concerned with the asymptotic analysis of nonbinary spatially-coupled low-density parity-check (SC-LDPC) ensembles defined over GL$\left(2^{m}\right)$ (the general linear group of degree $m$ over GF$\left(2\right)$). Our purpose is to prove threshold saturation when the transmission takes place on the binary erasure channel (BEC). To this end, we establish the duality rule for entropy for nonbinary variable-node (VN) and check-node (CN) convolutional operators to accommodate the nonbinary density evolution (DE) analysis. Based on this, we construct the explicit forms of th…
A general inner bound is given for the discrete memoryless broadcast channel with an arbitrary number of users and general message sets, a setting that accounts for the most general form of concurrent groupcasting, with messages intended for any set of subsets of receivers. Achievability is based on superposition coding and rate-splitting without and with binning, where each receiver jointly decodes both its desired messages as well as the partial interference assigned to it via rate-splitting. The proof of achievability builds on the techniques for the description and analysis of superpositio…
Spatially-coupled low-density parity-check (LDPC) codes, which were first introduced as LDPC convolutional codes, have been shown to exhibit excellent performance under low-complexity belief-propagation decoding. This phenomenon is now termed threshold saturation via spatial coupling. Spatially-coupled codes have been successfully applied in numerous areas. In particular, it was proven that spatially-coupled regular LDPC codes universally achieve capacity over the class of binary memoryless symmetric (BMS) channels under belief-propagation decoding. Recently, potential functions have been used…
Sparse superposition (SS) codes were originally proposed as a capacity-achieving communication scheme over the additive white Gaussian noise channel (AWGNC) [1]. Very recently, it was discovered that these codes are universal, in the sense that they achieve capacity over any memoryless channel under generalized approximate message-passing (GAMP) decoding [2], although this decoder has never been stated for SS codes. In this contribution we introduce the GAMP decoder for SS codes, we confirm empirically the universality of this communication scheme through its study on various channels and we p…
The cognitive interference channel is an interference channel in which one transmitter is non-causally provided with the message of the other transmitter. This channel model has been extensively studied in the past years and capacity results for certain classes of channels have been proved. In this paper we present new inner and outer bounds for the capacity region of the cognitive interference channel as well as new capacity results. Previously proposed outer bounds are expressed in terms of auxiliary random variables for which no cardinality constraint is known. Consequently it is not possib…
Recently, it has been observed that terminated low-density-parity-check (LDPC) convolutional codes (or spatially-coupled codes) appear to approach capacity universally across the class of binary memoryless channels. This is facilitated by the "threshold saturation" effect whereby the belief-propagation (BP) threshold of the spatially-coupled ensemble is boosted to the maximum a-posteriori (MAP) threshold of the underlying constituent ensemble. In this paper, we consider the universality of spatially-coupled codes over intersymbol-interference (ISI) channels under joint iterative decoding. More…
Low-density parity-check (LDPC) convolutional codes (or spatially-coupled codes) have been shown to approach capacity on the binary erasure channel (BEC) and binary-input memoryless symmetric channels. The mechanism behind this spectacular performance is the threshold saturation phenomenon, which is characterized by the belief-propagation threshold of the spatially-coupled ensemble increasing to an intrinsic noise threshold defined by the uncoupled system. In this paper, we present a simple proof of threshold saturation that applies to a broad class of coupled scalar recursions. The conditions…
We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded and transmitted over two independent binary memoryless symmetric channels. Each channel capacity is assumed to be characterized by a single parameter which is not known at the transmitter. The receiver has knowledge of both the source correlation and the channel parameters. We call a system universal if it retains near-capacity performance without channel knowledge at the transmitter. Kudekar et al. recently showed that terminated low-density parity-check (LDPC) convolutional codes (a.k.a. spatially-co…
A new achievability scheme for the compound channel with discrete memoryless (DM) state noncausally available at the encoder is established. Achievability is proved using superposition coding, Marton coding, joint typicality encoding, and indirect decoding. The scheme is shown to achieve strictly higher rate than the straightforward extension of the Gelfand-Pinsker coding scheme for a single DMC with DM state, and is optimal for some classes of channels.
Low-density parity-check (LDPC) convolutional codes (or spatially-coupled codes) were recently shown to approach capacity on the binary erasure channel (BEC) and binary-input memoryless symmetric channels. The mechanism behind this spectacular performance is now called threshold saturation via spatial coupling. This new phenomenon is characterized by the belief-propagation threshold of the spatially-coupled ensemble increasing to an intrinsic noise threshold defined by the uncoupled system. In this paper, we present a simple proof of threshold saturation that applies to a wide class of coupled…
This paper considers the cognitive interference channel (CIC) with two transmitters and two receivers, in which the cognitive transmitter non-causally knows the message and codeword of the primary transmitter. We first introduce a discrete memoryless more capable CIC, which is an extension to the more capable broadcast channel (BC). Using superposition coding, we propose an inner bound and an outer bound on its capacity region. The outer bound is also valid when the primary user is under strong interference. For the Gaussian CIC, this outer bound applies for $|a| \geq 1 $, where $a$ is the gai…
Low-density parity-check (LDPC) convolutional codes have been shown to exhibit excellent performance under low-complexity belief-propagation decoding [1], [2]. This phenomenon is now termed threshold saturation via spatial coupling. The underlying principle behind this appears to be very general and spatially-coupled (SC) codes have been successfully applied in numerous areas. Recently, SC regular LDPC codes have been proven to achieve capacity universally, over the class of binary memoryless symmetric (BMS) channels, under belief-propagation decoding [3], [4]. In [5], [6], potential functions…
We consider transmission of two independent and separately encoded sources over a two-user binary-input Gaussian multiple-access channel. The channel gains are assumed to be unknown at the transmitter and the goal is to design an encoder-decoder pair that achieves reliable communication for all channel gains where this is theoretically possible. We call such a system \emph{universal} with respect to the channel gains. Kudekar et al. recently showed that terminated low-density parity-check convolutional codes (a.k.a. spatially-coupled low-density parity-check ensembles) have belief-propagation …
It was recently shown that spatial coupling of individual low-density parity-check codes improves the belief-propagation threshold of the coupled ensemble essentially to the maximum a posteriori threshold of the underlying ensemble. We study the performance of spatially coupled low-density generator-matrix ensembles when used for transmission over binary-input memoryless output-symmetric channels. We show by means of density evolution that the threshold saturation phenomenon also takes place in this setting. Our motivation for studying low-density generator-matrix codes is that they can easily…
We consider spatially coupled code ensembles over a multiple access channel. Convolutional LDPC ensembles are one instance of spatially coupled codes. It was shown recently that, for transmission over the binary erasure channel, this coupling of individual code ensembles has the effect of increasing the belief propagation threshold of the coupled ensembles to the maximum a-posteriori threshold of the underlying ensemble. In this sense, spatially coupled codes were shown to be capacity achieving. It was observed, empirically, that these codes are universal in the sense that they achieve perform…
This paper presents polar coding schemes for the 2-user discrete memoryless broadcast channel (DM-BC) which achieve Marton's region with both common and private messages. This is the best achievable rate region known to date, and it is tight for all classes of 2-user DM-BCs whose capacity regions are known. To accomplish this task, we first construct polar codes for both the superposition as well as the binning strategy. By combining these two schemes, we obtain Marton's region with private messages only. Finally, we show how to handle the case of common information. The proposed coding scheme…
In this paper, we formulate a new problem to cope with the transmission of extra bits over an existing coded transmission link (referred to as coded payload link) without any cost of extra transmission energy or extra bandwidth. This is possible since a gap to the channel capacity typically exists for a practical code. A new concept, termed as accessible capacity, is introduced to specify the maximum rate at which the superposition transmission of extra bits is reliable and has a negligible effect on the performance of the coded payload link. For a binary-input output-symmetric (BIOS) memoryle…
Polar codes are introduced for discrete memoryless broadcast channels. For $m$-user deterministic broadcast channels, polarization is applied to map uniformly random message bits from $m$ independent messages to one codeword while satisfying broadcast constraints. The polarization-based codes achieve rates on the boundary of the private-message capacity region. For two-user noisy broadcast channels, polar implementations are presented for two information-theoretic schemes: i) Cover's superposition codes; ii) Marton's codes. Due to the structure of polarization, constraints on the auxiliary and…
This paper investigates polar codes for the additive white Gaussian noise (AWGN) channel. The scaling exponent $μ$ of polar codes for a memoryless channel $q_{Y|X}$ with capacity $I(q_{Y|X})$ characterizes the closest gap between the capacity and non-asymptotic achievable rates in the following way: For a fixed $\varepsilon \in (0, 1)$, the gap between the capacity $I(q_{Y|X})$ and the maximum non-asymptotic rate $R_n^*$ achieved by a length-$n$ polar code with average error probability $\varepsilon$ scales as $n^{-1/μ}$, i.e., $I(q_{Y|X})-R_n^* = Θ(n^{-1/μ})$. It is well known that the scalin…
We consider the problem of communication over a three user discrete memoryless interference channel ($3-$IC). The current known coding techniques for communicating over an arbitrary $3-$IC are based on message splitting, superposition coding and binning using independent and identically distributed (iid) random codebooks. In this work, we propose a new ensemble of codes - partitioned coset codes (PCC) - that possess an appropriate mix of empirical and algebraic closure properties. We develop coding techniques that exploit algebraic closure property of PCC to enable efficient communication over…
Recently, it was observed that spatially-coupled LDPC code ensembles approach the Shannon capacity for a class of binary-input memoryless symmetric (BMS) channels. The fundamental reason for this was attributed to a "threshold saturation" phenomena derived by Kudekar, Richardson and Urbanke. In particular, it was shown that the belief propagation (BP) threshold of the spatially coupled codes is equal to the maximum a posteriori (MAP) decoding threshold of the underlying constituent codes. In this sense, the BP threshold is saturated to its maximum value. Moreover, it has been empirically obser…
Channel polarization is a phenomenon in which a particular recursive encoding induces a set of synthesized channels from many instances of a memoryless channel, such that a fraction of the synthesized channels becomes near perfect for data transmission and the other fraction becomes near useless for this task. Mahdavifar and Vardy have recently exploited this phenomenon to construct codes that achieve the symmetric private capacity for private data transmission over a degraded wiretap channel. In the current paper, we build on their work and demonstrate how to construct quantum wiretap polar c…
