Probabilistic amplitude shaping (PAS) is a coded modulation strategy in which constellation shaping and channel coding are combined. PAS has attracted considerable attention in both wireless and optical communications. Achievable information rates (AIRs) of PAS have been investigated in the literature using Gallager's error exponent approach. In particular, it has been shown that PAS achieves the capacity of the additive white Gaussian noise channel (Böcherer, 2018). In this work, we revisit the capacity-achieving property of PAS and derive AIRs using weak typicality. Our objective is to provide alternative proofs based on random sign-coding arguments that are as constructive as possible. Accordingly, in our proofs, only some signs of the channel inputs are drawn from a random code, while the remaining signs and amplitudes are produced constructively. We consider both symbol-metric and bit-metric decoding.
For a layered probabilistic shaping (PS) scheme with a general decoding metric, an achievable rate is derived using Gallager's error exponent approach and the concept of achievable code rates is introduced. Several instances for specific decoding metrics are discussed, including bit-metric decoding, interleaved coded modulation, and hard-decision decoding. It is shown that important previously known achievable rates can also be achieved by layered PS. A practical instance of layered PS is the recently proposed probabilistic amplitude shaping (PAS).
Achievable information rates are used as a metric to design novel modulation formats via geometric shaping. The proposed geometrically shaped 256-ary constellation achieves SNR gains of up to 1.18 dB.
Dirty paper coding (DPC) refers to methods for pre-subtraction of known interference at the transmitter of a multiuser communication system. There are numerous applications for DPC, including coding for broadcast channels. Recently, lattice-based coding techniques have provided several designs for DPC. In lattice-based DPC, there are two codes - a convolutional code that defines a lattice used for shaping and an error correction code used for channel coding. Several specific designs have been reported in the recent literature using convolutional and graph-based codes for capacity-approaching s…
A probabilistic shaping method for multi-level coding (MLC) is presented, where the transmitted symbols are forced to have a shaped non-uniform distribution. It is shown that shaping only a single bit-level suffices to compensate for most of the shaping loss on the fading channels. A polar code based implementation of the proposed scheme is presented, where shaping is performed by using a precoder at the transmitter without increasing the decoding complexity. Simulation results show that performance improvements can be obtained compared to BICM- and MLC-based polar coding without shaping on Ra…
Achievable information rates (AIRs) of wideband optical communication systems using ~40 nm (~5 THz) EDFA and ~100 nm (~12.5 THz) distributed Raman amplification are estimated based on a first-order perturbation analysis. The AIRs of each individual channel have been evaluated for DP-64QAM, DP-256QAM, and DP-1024QAM modulation formats. The impact of full-field nonlinear compensation (FF-NLC) and probabilistically shaped constellations using a Maxwell-Boltzmann distribution were studied and compared to electronic dispersion compensation. It is found that a probabilistically shaped DP-1024QAM con…
The performance of bit-interleaved coded modulation (BICM) with bit shaping (i.e., non-equiprobable bit probabilities in the underlying binary code) is studied. For the Gaussian channel, the rates achievable with BICM and bit shaping are practically identical to those of coded modulation or multilevel coding. This identity holds for the whole range of values of signal-to-noise ratio. Moreover, the random coding error exponent of BICM significantly exceeds that of multilevel coding and is very close to that of coded modulation.
We propose two low-complexity lattice code constructions that have competitive coding and shaping gains. The first construction, named systematic Voronoi shaping, maps short blocks of integers to the dithered Voronoi integers, which are dithered integers that are uniformly distributed over the Voronoi region of a low-dimensional shaping lattice. Then, these dithered Voronoi integers are encoded using a high-dimensional lattice retaining the same shaping and coding gains of low and high-dimensional lattices. A drawback to this construction is that there is no isomorphism between the underlying …
We are interested in how to best communicate a (usually real valued) source to a number of destinations (sinks) over a network with capacity constraints in a collective fidelity metric over all the sinks, a problem which we call joint network-source coding. Unlike the lossless network coding problem, lossy reconstruction of the source at the sinks is permitted. We make a first attempt to characterize the set of all distortions achievable by a set of sinks in a given network. While the entire region of all achievable distortions remains largely an open problem, we find a large, non-trivial subs…
In this paper, we investigate the additive Gaussian noise channel with noisy feedback. We consider the setup of linear coding of the feedback information and Gaussian signaling of the message (i.e. Cover-Pombra Scheme). Then, we derive the upper and lower bounds on the largest achievable rate for this setup. We show that these two bounds can be obtained by solving two convex optimization problems. Finally, we present some simulations and discussion.
The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Analogously, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system to establish the source coding theorem in the second- order sense. The Slepian-Wolf coding problem for correlated sources is one of typical problems in the multi-terminal i…
A multiple-descriptions (MD) coding strategy is proposed and an inner bound to the achievable rate-distortion region is derived. The scheme utilizes linear codes. It is shown in two different MD set-ups that the linear coding scheme achieves a larger rate-distortion region than previously known random coding strategies. Furthermore, it is shown via an example that the best known random coding scheme for the set-up can be improved by including additional randomly generated codebooks.
This is a companion technical report of the main manuscript "Physical-Layer Multicasting by Stochastic Transmit Beamforming and Alamouti Space-Time Coding". The report serves to give detailed derivations of the achievable rate functions encountered in the main manuscript, which are too long to be included in the latter. In addition, more simulation results are presented to verify the viability of the multicast schemes developed in the main manuscript.
We consider the problem of multiple descriptions (MD) source coding and propose new coding strategies involving both unstructured and structured coding layers. Previously, the most general achievable rate-distortion (RD) region for the $l$-descriptions problem was the Combinatorial Message Sharing with Binning (CMSB) region. The CMSB scheme utilizes unstructured quantizers and unstructured binning. In the first part of the paper, we show that this strategy can be improved upon using more general unstructured quantizers and a more general unstructured binning method. In the second part, structu…
Unequal transition probabilities between input and output symbols, input power constraints, or input symbols of unequal durations can lead to non-uniform capacity achieving input distributions for communication channels. Using uniform input distributions reduces the achievable rate, which is called the shaping gap. Gallager's idea for reliable communication with zero shaping gap is to do encoding, matching, and jointly decoding and dematching. In this work, a scheme is proposed that consists in matching, encoding, decoding, and dematching. Only matching is channel specific whereas coding is no…
Data shaping is a coding technique that has been proposed to increase the lifetime of flash memory devices. Several data shaping codes have been described in recent work, including endurance codes and direct shaping codes for structured data. In this paper, we study information-theoretic properties of a general class of data shaping codes and prove a separation theorem stating that optimal data shaping can be achieved by the concatenation of optimal lossless compression with optimal endurance coding. We also determine the expansion factor that minimizes the total wear cost. Finally, we analyze…
In this work, lossy distributed compression of pairs of correlated sources is considered. Conventionally, Shannon's random coding arguments -- using randomly generated unstructured codebooks whose blocklength is taken to be asymptotically large -- are used to derive achievability results. However, it was recently observed that in various multi-terminal communications scenarios, using random codes with constant finite blocklength may lead to improved achievable regions compared to the conventional approach. In other words, in some network communication scenarios, there is a finite optimal value…
We investigate the achievable rate of data transmission from sources to sinks through a multiple-relay network. We study achievable rates for omniscient coding, in which all nodes are considered in the coding design at each node. We find that, when maximizing the achievable rate, not all nodes need to ``cooperate'' with all other nodes in terms of coding and decoding. This leads us to suggest a constrained network, whereby each node only considers a few neighboring nodes during encoding and decoding. We term this myopic coding and calculate achievable rates for myopic coding. We show by exampl…
Signal shaping is vital to approach Shannon's capacity, yet it is challenging to implement at very high speeds. For example, probabilistic shaping often requires arithmetic coding to realize the target distribution. Geometric shaping requires look-up tables to store the constellation points. In this paper, we propose a four-dimensional amplitude coding (4D-AC) geometrical shaper architecture. The proposed architecture can generate in real time geometrically shaped 4D formats via simple logic circuit operations and two conventional quadrature amplitude modulation (QAM) modulators. This paper de…
Determining the achievable rate region for networks using routing, linear coding, or non-linear coding is thought to be a difficult task in general, and few are known. We describe the achievable rate regions for four interesting networks (completely for three and partially for the fourth). In addition to the known matrix-computation method for proving outer bounds for linear coding, we present a new method which yields actual characteristic-dependent linear rank inequalities from which the desired bounds follow immediately.
The interference channel with degraded message sets (IC-DMS) refers to a communication model in which two senders attempt to communicate with their respective receivers simultaneously through a common medium, and one of the senders has complete and a priori (non-causal) knowledge about the message being transmitted by the other. A coding scheme that collectively has advantages of cooperative coding, collaborative coding, and dirty paper coding, is developed for such a channel. With resorting to this coding scheme, achievable rate regions of the IC-DMS in both discrete memoryless and Gaussian c…
We study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure/list option at the decoder, in the random coding regime. Specifically, we harness a universal version of Forney's classical erasure/list decoder developed in earlier studies, which is based on the competitive minimax methodology, and guarantees universal achievability of a certain fraction of the optimum random coding error exponents. In this paper, we derive an exact single-letter expression for the maximum achievable fraction. Examples are given in which the maximal achievable fract…
In this paper, we use entropy functions to characterise the set of rate-capacity tuples achievable with either zero decoding error, or vanishing decoding error, for general network coding problems. We show that when sources are colocated, the outer bound obtained by Yeung, A First Course in Information Theory, Section 15.5 (2002) is tight and the sets of zero-error achievable and vanishing-error achievable rate-capacity tuples are the same. We also characterise the set of zero-error and vanishing-error achievable rate capacity tuples for network coding problems subject to linear encoding const…
This paper studies Gaussian Two-Way Relay Channel where two communication nodes exchange messages with each other via a relay. It is assumed that all nodes operate in half duplex mode without any direct link between the communication nodes. A compress-and-forward relaying strategy using nested lattice codes is first proposed. Then, the proposed scheme is improved by performing a layered coding : a common layer is decoded by both receivers and a refinement layer is recovered only by the receiver which has the best channel conditions. The achievable rates of the new scheme are characterized and …
One of the biggest criticisms of the Set Shaping Theory is the lack of a practical application. This is due to the difficulty of its application. In fact, to apply this technique from an experimental point of view we must use a table that defines the correspondences between two sets. However, this approach is not usable in practice, because the table has A^N elements, with A number of symbols and N length of the message to be encoded. Consequently, these tables can be implemented in a program only when A and N have a low value. Unfortunately, in these cases, there are no compression algorithms…
A new polar coding scheme for higher order modulation is presented. The proposed scheme is based on multi-level coding (MLC) with natural labeling, where the bit-level corresponding to the sign-bit is generated in dependence on the previous bit-levels, such that the modulated symbols are distributed according to a target non-uniform distribution resulting in a shaping gain. This is realized by replacing the polar encoder in the sign-bit level by a successive cancellation (SC) decoder, such that the overall complexity is not increased compared to a conventional MLC scheme with polar codes. Nume…
The recent increase in number of wireless devices has been driven by the growing markets of smart homes and the Internet of Things (IoT). As a result, expanding and/or efficient utilization of the radio frequency (RF) spectrum is critical to accommodate such an increase in wireless bandwidth. Alternatively, recent free-space optical (FSO) communication technologies have demonstrated the feasibility of building WiFO, a high capacity indoor wireless network using the femtocell architecture. Since FSO transmission does not interfere with the RF signals, such a system can be integrated with the cu…
A noncoherent two-way relaying system is developed using physical-layer network coding for improved throughput over conventional relaying in a fading channel. Energy-efficient noncoherent operation is achieved using multitone frequency shift keying (FSK). A novel soft-output demodulator is developed for the relay, and corresponding achievable exchange rates are found for Rayleigh fading and AWGN channels. Bit-error rate performance approaching the achievable rate is realized using a capacity-approaching channel code and a receiver architecture that iterates between demodulation and channel dec…
We derive a general formula of the minimum achievable rate for fixed-to-variable length coding with a regular cost function by allowing the error probability up to a constant $\varepsilon$. For a fixed-to-variable length code, we call the set of source sequences that can be decoded without error the dominant set of source sequences. For any two regular cost functions, it is revealed that the dominant set of source sequences for a code attaining the minimum achievable rate with a cost function is also the dominant set for a code attaining the minimum achievable rate with the other cost function…
In this paper, we study the asymptotic performance of Abelian group codes for the lossy source coding problem for arbitrary discrete (finite alphabet) memoryless sources as well as the channel coding problem for arbitrary discrete (finite alphabet) memoryless channels. For the source coding problem, we derive an achievable rate-distortion function that is characterized in a single-letter information-theoretic form using the ensemble of Abelian group codes. When the underlying group is a field, it simplifies to the symmetric rate-distortion function. Similarly, for the channel coding problem, w…
This paper addresses the fundamental characteristics of information exchange via multihop network coding over two-way relaying in a wireless ad hoc network. The end-to-end rate regions achieved by time-division multihop (TDMH), MAC-layer network coding (MLNC) and PHY-layer network coding (PLNC) are first characterized. It is shown that MLNC does not always achieve better rates than TDMH, time sharing between TDMH and MLNC is able to achieve a larger rate region, and PLNC dominates the rate regions achieved by TDMH and MLNC. An opportunistic scheduling algorithm for MLNC and PLNC is then propos…
A new ensemble of structured codes is introduced. These codes are called Quasi Linear Codes (QLC). The QLC's are constructed by taking subsets of linear codes. They have a looser structure compared to linear codes and are not closed under addition. We argue that these codes provide gains in terms of achievable Rate-Distortions (RD) in different multi-terminal source coding problems. We derive the necessary covering bounds for analyzing the performance of QLC's. We then consider the Multiple-Descriptions (MD) problem, and prove through an example that the application of QLC's gives an improved …
Proving capacity for networks without feedback or cooperation usually involves two fundamental random coding techniques: superposition coding and binning. Although conceptually very different, these two techniques often achieve the same performance, suggesting an underlying similarity. In this correspondence we propose a new random coding technique that generalizes superposition coding and binning and provides new insight on relationship among the two With this new theoretical tool, we derive new achievable regions for three classical information theoretical models: multi-access channel, broad…
In this paper, we propose an achievable rate region for discrete memoryless interference channels with conferencing at the transmitter side. We employ superposition block Markov encoding, combined with simultaneous superposition coding, dirty paper coding, and random binning to obtain the achievable rate region. We show that, under respective conditions, the proposed achievable region reduces to Han and Kobayashi achievable region for interference channels, the capacity region for degraded relay channels, and the capacity region for the Gaussian vector broadcast channel. Numerical examples for…
The distributed source coding problem is considered when the sensors, or encoders, are under Byzantine attack; that is, an unknown number of sensors have been reprogrammed by a malicious intruder to undermine the reconstruction at the fusion center. Three different forms of the problem are considered. The first is a variable-rate setup, in which the decoder adaptively chooses the rates at which the sensors transmit. An explicit characterization of the variable-rate minimum achievable sum rate is stated, given by the maximum entropy over the set of distributions indistinguishable from the true …
Consider the following information theoretic setup wherein independent codebooks of N correlated random variables are generated according to their respective marginals. The problem of determining the conditions on the rates of codebooks to ensure the existence of at least one codeword tuple which is jointly typical with respect to a given joint density (called the multivariate covering lemma) has been studied fairly well and the associated rate regions have found applications in several source coding scenarios. However, several multiterminal source coding applications, such as the general mult…
A single-letter achievable rate region is proposed for the two-receiver discrete memoryless broadcast channel with generalized feedback. The coding strategy involves block-Markov superposition coding, using Marton's coding scheme for the broadcast channel without feedback as the starting point. If the message rates in the Marton scheme are too high to be decoded at the end of a block, each receiver is left with a list of messages compatible with its output. Resolution information is sent in the following block to enable each receiver to resolve its list. The key observation is that the resolut…
We present an achievable rate region for the general three user discrete memoryless broadcast channel, based on nested coset codes. We characterize 3-to-1 discrete broadcast channels, a class of broadcast channels for which the best known coding technique\footnote{We henceforth refer to this as Marton's coding for three user discrete broadcast channel.}, which is obtained by a natural generalization of that proposed by Marton for the general two user discrete broadcast channel, is strictly sub-optimal. In particular, we identify a novel 3-to-1 discrete broadcast channel for which Marton's codi…
We present a superposition coding scheme for communication over a network, which combines partial decode and forward and noisy network coding. This hybrid scheme is termed as superposition noisy network coding. The scheme is designed and analyzed for single relay channel, single source multicast network and multiple source multicast network. The achievable rate region is determined for each case. The special cases of Gaussian single relay channel and two way relay channel are analyzed for superposition noisy network coding. The achievable rate of the proposed scheme is higher than the existing…
We consider the problem of communication over a three user discrete memoryless interference channel ($3-$IC). The current known coding techniques for communicating over an arbitrary $3-$IC are based on message splitting, superposition coding and binning using independent and identically distributed (iid) random codebooks. In this work, we propose a new ensemble of codes - partitioned coset codes (PCC) - that possess an appropriate mix of empirical and algebraic closure properties. We develop coding techniques that exploit algebraic closure property of PCC to enable efficient communication over…
In this paper, we study the state-dependent two-user interference channel, where the state information is non-causally known at both transmitters but unknown to either of the receivers. We propose two coding schemes for the discrete memoryless case: simultaneous encoding for the sub-messages in the first one and superposition encoding in the second one, both with rate splitting and Gel'fand-Pinsker coding. The corresponding achievable rate regions are established.
Designs and methods for nested lattice codes using Construction D' lattices for coding and convolutional code lattices for shaping are described. Two encoding methods and a decoding algorithm for Construction D' coding lattices that can be used with shaping lattices for power-constrained channels are given. We construct nested lattice codes with good coding properties, a high shaping gain, and low-complexity encoding and decoding. Convolutional code generator polynomials for Construction A lattices with the greatest shaping gain are given, as a result of an extensive search. It is shown that r…
In this paper, it is shown that for any given single-hop communication network with two receivers, splitting messages into more than two sub-messages in a random coding scheme is redundant. To this end, the Broadcast Channel with Cognitive Relays (BCCR) is considered. A novel achievability scheme is designed for this network. Our achievability design is derived by a systematic combination of the best known achievability schemes for the basic building blocks included in the network: the Han-Kobayashi scheme for the two-user interference channel and the Marton coding scheme for the broadcast cha…
One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Renyi entropy and smooth max Renyi divergence. Asymptotically (in the limit of large block lengths) this region is quantified in terms of spectral-sup conditional entropy rate and spectral- sup mutual information rate. In particular, it coincides with the rate region derived in the limit of unlimited independent and identically distributed copies of the sources.
Motivated by linear network coding, communication channels perform linear operation over finite fields, namely linear operator channels (LOCs), are studied in this paper. For such a channel, its output vector is a linear transform of its input vector, and the transformation matrix is randomly and independently generated. The transformation matrix is assumed to remain constant for every T input vectors and to be unknown to both the transmitter and the receiver. There are NO constraints on the distribution of the transformation matrix and the field size. Specifically, the optimality of subspace …
This paper proves the separation between source-network coding and channel coding in networks of noisy, discrete, memoryless channels. We show that the set of achievable distortion matrices in delivering a family of dependent sources across such a network equals the set of achievable distortion matrices for delivering the same sources across a distinct network which is built by replacing each channel by a noiseless, point-to-point bit-pipe of the corresponding capacity. Thus a code that applies source-network coding across links that are made almost lossless through the application of independ…
The problem of mismatched decoding for discrete memoryless channels is addressed. A mismatched cognitive multiple-access channel is introduced, and an inner bound on its capacity region is derived using two alternative encoding methods: superposition coding and random binning. The inner bounds are derived by analyzing the average error probability of the code ensemble for both methods and by a tight characterization of the resulting error exponents. Random coding converse theorems are also derived. A comparison of the achievable regions shows that in the matched case, random binning performs a…
Probabilistic amplitude shaping (PAS) is on track to become the de facto coded modulation standard for communication systems aiming to operate close to channel capacity at high transmission rates. The essential component of PAS that breeds this widespread interest is the amplitude shaping block, through which the channel input distribution is controlled. This block is responsible for converting bit strings into amplitude sequences with certain properties, e.g., fixed composition, limited energy, limited energy variation, etc. Recently, band-trellis enumerative sphere shaping (B-ESS) was introd…
The problem of maximum rate achievable with analog network coding for a unicast communication over a layered relay network with directed links is considered. A relay node performing analog network coding scales and forwards the signals received at its input. Recently this problem has been considered under certain assumptions on per node scaling factor and received SNR. Previously, we established a result that allows us to characterize the optimal performance of analog network coding in network scenarios beyond those that can be analyzed using the approaches based on such assumptions. The key c…
Linear coding schemes have been the main choice of coding for the additive white Gaussian noise broadcast channel (AWGN-BC) with noiseless feedback in the literature. The achievable rate regions of these schemes go well beyond the capacity region of the AWGN-BC without feedback. In this paper, a concatenating coding design for the $K$-user AWGN-BC with noisy feedback is proposed that relies on linear feedback schemes to achieve rate tuples outside the no-feedback capacity region. Specifically, a linear feedback code for the AWGN-BC with noisy feedback is used as an inner code that creates an e…
A new channel coding approach was proposed in [1] for random multiple access communication over the discrete-time memoryless channel. The coding approach allows users to choose their communication rates independently without sharing the rate information among each other or with the receiver. The receiver will either decode the message or report a collision depending on whether reliable message recovery is possible. It was shown that, asymptotically as the codeword length goes to infinity, the set of communication rates supporting reliable message recovery can be characterized by an achievable …
An index coding problem with $n$ messages has symmetric rate $R$ if all $n$ messages can be conveyed at rate $R$. In a recent work, a class of index coding problems for which symmetric rate $\frac{1}{3}$ is achievable was characterised using special properties of the side-information available at the receivers. In this paper, we show a larger class of index coding problems (which includes the previous class of problems) for which symmetric rate $\frac{1}{3}$ is achievable. In the process, we also obtain a stricter necessary condition for rate $\frac{1}{3}$ feasibility than what is known in lit…
A new achievable rate for bit-metric decoding (BMD) is derived using random coding arguments. The rate expression can be evaluated for any input distribution, and in particular the bit-levels of binary input labels can be stochastically dependent. Probabilistic shaping with dependent bit-levels (shaped BMD), shaping of independent bit-levels (bit-shaped BMD) and uniformly distributed independent bit-levels (uniform BMD) are evaluated on the additive white Gaussian noise (AWGN) channel with Gray labeled bipolar amplitude shift keying (ASK). For 32-ASK at a rate of 3.8 bits/channel use, the gap …
The joint design of input constellation and low-density parity-check (LDPC) codes to approach the symmetric capacity of the two-user Gaussian multiple access channel is studied. More specifically, multilevel coding is employed at each user to construct a high-order input constellation and the constellations of the users are jointly designed so as to maximize the multiuser shaping gain. At the receiver, each layer of the multilevel coding is jointly decoded among users, while successive cancellation is employed across layers. The LDPC code employed by each user in each layer is designed using E…
In this paper, we first study a two-user interference channel with generalized feedback. We establish an inner bound on its capacity region. The coding scheme that we employ for the inner bound is based on an appropriate combination of Han-Kobayash rate splitting and compress-and-forward at the senders. Each sender compresses the channel output that is observes using a compression scheme that is à-la Lim et al. noisy network coding and Avestimeher et al. quantize-map-and-forward. Next, we study an injective deterministic model in which the senders obtain output feedback only intermittently. Sp…
This paper proposes a novel achievable scheme for the index problem and applies it to the caching problem. Index coding and caching are noiseless broadcast channel problems where receivers have message side information.In the index coding problem the side information sets are fixed, while in the caching problem the side information sets correspond the cache contents, which are under the control of the system designer. The proposed index coding scheme, based on distributed source coding and non-unique decoding,is shown to strictly enlarge the rate region achievable by composite coding.The novel…
