We consider the topic of universal decoding with a decoder that does not have direct access to the codebook, but only to noisy versions of the various randomly generated codewords, a problem motivated by biometrical identification systems. Both the source that generates the original (clean) codewords, and the channel that corrupts them in generating the noisy codewords, as well as the main channel for communicating the messages, are all modeled by non-unifilar, finite-state systems (hidden Markov models). As in previous works on universal decoding, here too, the average error probability of our proposed universal decoder is shown to be as small as that of the optimal maximum likelihood (ML) decoder, up to a multiplicative factorthat is a sub-exponential function of the block length. It therefore has the same error exponent, whenever the ML decoder has a positive error exponent. The universal decoding metric is based on Lempel-Ziv (LZ) incremental parsing of each noisy codeword jointly with the given channel output vector, but this metric is somewhat different from the one proposed in earlier works on universal decoding for finite-state channels, by Ziv (1985) and by Lapidoth and Ziv (1998). The reason for the difference is that here, unlike in those earlier works, the probability distribution that governs the (noisy) codewords is, in general, not uniform across its support. This non-uniformity of the codeword distribution also makes our derivation more challenging. Another reason for the more challenging analysis is the fact that the effective induced channel between the noisy codeword of the transmitted message and the main channel output is not a finite-state channel in general.
We consider an ensemble of constant composition codes that are subsets of linear codes: while the encoder uses only the constant-composition subcode, the decoder operates as if the full linear code was used, with the motivation of simultaneously benefiting both from the probabilistic shaping of the channel input and from the linear structure of the code. We prove that the codebook mismatch can be fully compensated by using a mismatched additive decoding metric that achieves the random coding error exponent of (non-linear) constant composition codes. As the coding rate tends to the mutual infor…
Shaping gain is attained in schemes where a shaped subcode is chosen from a larger codebook by a codeword selection process. This includes the popular method of Trellis Shaping (TS), originally proposed by Forney for average power reduction. The decoding process of such schemes is mismatched, since it is aware of only the large codebook. This study models such schemes by a random code construction and derives achievable bounds on the transmission rate under matched and mismatched decoding. For matched decoding the bound is obtained using a modified asymptotic equipartition property (AEP) theor…
The differential encoding/decoding setup introduced by Kiran et al, Oggier et al and Jing et al for wireless relay networks that use codebooks consisting of unitary matrices is extended to allow codebooks consisting of scaled unitary matrices. For such codebooks to be used in the Jing-Hassibi protocol for cooperative diversity, the conditions that need to be satisfied by the relay matrices and the codebook are identified. A class of previously known rate one, full diversity, four-group encodable and four-group decodable Differential Space-Time Codes (DSTCs) is proposed for use as Distributed D…
This paper conceives a novel sparse code multiple access (SCMA) codebook design which is motivated by the strong need for providing ultra-low decoding complexity and good error performance in downlink Internet-of-things (IoT) networks, in which a massive number of low-end and low-cost IoT communication devices are served. By focusing on the typical Rician fading channels, we analyze the pair-wise probability of superimposed SCMA codewords and then deduce the design metrics for multi-dimensional constellation construction and sparse codebook optimization. For significant reduction of the decodi…
Consider a receiver in a multi-user network that wishes to decode several messages. Simultaneous joint typicality decoding is one of the most powerful techniques for determining the fundamental limits at which reliable decoding is possible. This technique has historically been used in conjunction with random i.i.d. codebooks to establish achievable rate regions for networks. Recently, it has been shown that, in certain scenarios, nested linear codebooks in conjunction with "single-user" or sequential decoding can yield better achievable rates. For instance, the compute-forward problem examines…
The differential encoding/decoding setup introduced by Kiran et al, Oggier-Hassibi and Jing-Jafarkhani for wireless relay networks that use codebooks consisting of unitary matrices is extended to allow codebooks consisting of scaled unitary matrices. For such codebooks to be usable in the Jing-Hassibi protocol for cooperative diversity, the conditions involving the relay matrices and the codebook that need to be satisfied are identified. Using the algebraic framework of extended Clifford algebras, a new class of Distributed Differential Space-Time Codes satisfying these conditions for power of…
In the presence of multiple senders, one of the simplest decoding strategies that can be employed by a receiver is successive decoding. In a successive decoding strategy, the receiver decodes the messages one at a time using the knowledge of the previously decoded messages as side information. Recently, there have been two separate attempts to construct codes for the interference channel using successive decoding based on the idea of rate-splitting. In this note, we highlight a difficulty that arises when a rate-splitting codebook is to be decoded by multiple receivers. The main issue is that …
We study the second-order asymptotics of information transmission using random Gaussian codebooks and nearest neighbor (NN) decoding over a power-limited stationary memoryless additive non-Gaussian noise channel. We show that the dispersion term depends on the non-Gaussian noise only through its second and fourth moments, thus complementing the capacity result (Lapidoth, 1996), which depends only on the second moment. Furthermore, we characterize the second-order asymptotics of point-to-point codes over $K$-sender interference networks with non-Gaussian additive noise. Specifically, we assume …
We consider a channel-independent decoder which is for i.i.d. random codes what the maximum mutual-information decoder is for constant composition codes. We show that this decoder results in exactly the same i.i.d. random coding error exponent and almost the same correct-decoding exponent for a given codebook distribution as the maximum-likelihood decoder. We propose an algorithm for computation of the optimal correct-decoding exponent which operates on the corresponding expression for the channel-independent decoder. The proposed algorithm comes in two versions: computation at a fixed rate an…
In this paper, we consider the performance of the Noisy Gradient Descent Bit Flipping (NGDBF) algorithm under re-decoding of failed frames. NGDBF is a recent algorithm that uses a non-deterministic gradient descent search to decode low-density parity check (LDPC) codes. The proposed re-decode procedure obtains improved performance because the perturbations are independent at each re-decoding phase, therefore increasing the likelihood of successful decoding. We examine the benefits of re-decoding for an LDPC code from the IEEE 802.3an standard, and find that only a small fraction of re-decoded …
The achievable and converse regions for sparse representation of white Gaussian noise based on an overcomplete dictionary are derived in the limit of large systems. Furthermore, the marginal distribution of such sparse representations is also inferred. The results are obtained via the Replica method which stems from statistical mechanics. A direct outcome of these results is the introduction of sharp threshold for $\ell_{0}$-norm decoding in noisy compressed sensing, and its mean-square error for underdetermined Gaussian vector channels.
In this paper, we focus on the demodulation/decoding of the complex modulations/codes that approach the Shannon capacity. Theoretically, the maximum likelihood (ML) algorithm can achieve the optimal error performance whereas it has $\mathcal{O}(2^k)$ demodulation/decoding complexity with $k$ denoting the number of information bits. Recent progress in deep learning provides a new direction to tackle the demodulation and the decoding. The purpose of this paper is to analyze the feasibility of the neural network to demodulate/decode the complex modulations/codes close to the Shannon capacity and …
In multi-user information theory it is often assumed that every node in the network possesses all codebooks used in the network. This assumption is however impractical in distributed ad-hoc and cognitive networks. This work considers the two- user Gaussian Interference Channel with one Oblivious Receiver (G-IC-OR), i.e., one receiver lacks knowledge of the interfering cookbook while the other receiver knows both codebooks. We ask whether, and if so how much, the channel capacity of the G-IC- OR is reduced compared to that of the classical G-IC where both receivers know all codebooks. Intuitive…
A new class of spherical codes is constructed by selecting a finite subset of flat tori from a foliation of the unit sphere S^{2L-1} of R^{2L} and designing a structured codebook on each torus layer. The resulting spherical code can be the image of a lattice restricted to a specific hyperbox in R^L in each layer. Group structure and homogeneity, useful for efficient storage and decoding, are inherited from the underlying lattice codebook. A systematic method for constructing such codes are presented and, as an example, the Leech lattice is used to construct a spherical code in R^{48}. Upper an…
This paper studies channel coding for the discrete memoryless multiple-access channel with a given (possibly suboptimal) decoding rule. A multi-letter successive decoding rule depending on an arbitrary non-negative decoding metric is considered, and achievable rate regions and error exponents are derived both for the standard MAC (independent codebooks), and for the cognitive MAC (one user knows both messages) with superposition coding. In the cognitive case, the rate region and error exponent are shown to be tight with respect to the ensemble average. The rate regions are compared with those …
The problem of mismatched decoding with an additive metric $q$ for a discrete memoryless channel $W$ is addressed. The "product-space" improvement of the random coding lower bound on the mismatch capacity, $C_q^{(\infty)}(W)$, was introduced by Csiszár and Narayan. We study two kinds of decoders. The {\it $δ$-margin mismatched decoder} outputs a message whose metric with the channel output exceeds that of all the other codewords by at least $δ$. The {\it $τ$-threshold decoder} outputs a single message whose metric with the channel output exceeds a threshold $τ$. Both decoders declare an error …
The "weak" relay-eavesdropper channel was first studied by Lai and El Gamal, whose achievable scheme introduced noise forwarding (NF) and used backward decoding. We suggest a novel sliding window decoding scheme with a two block decoding delay, where the relay uses compress-forward with Wyner-Ziv (WZ) binning but does not use NF. Wireless engineers will welcome the reduced decoding delay. Sliding window decoding mandates multiblock equivocation calculations; dispensing with NF enables it. We identify nine regimes and develop a case-by-case choice of relay channel codebook and WZ bin sizes to m…
A noisy entropy inequality for boolean functions by Samorodnitsky is applied to binary codes. It is shown that a binary code that achieves capacity on the binary erasure channel admits optimal list size for list decoding on some binary symmetric channels (in a regime where this optimal list size is exponentially large).
The topic of this study is the outage behavior of multiple-antenna slow fading channels with quantized feedback and partial power control. A fixed-rate communication system is considered. It is known from the literature that with error-free feedback, the outage-optimal quantizer for power control has a circular structure. Moreover, the diversity gain of the system increases polynomially with the cardinality of the power control codebook. Here, a similar system is studied, but when the feedback link is error-prone. We prove that in the high-SNR regime, the optimal quantizer structure with noisy…
We address noisy message-passing decoding of lowdensity parity-check (LDPC) codes over additive white Gaussian noise channels. Message-passing decoders in which certain processing units iteratively exchange messages are common for decoding LDPC codes. The exchanged messages are in general subject to internal noise in hardware implementation of these decoders. We model the internal decoder noise as additive white Gaussian noise (AWGN) degrading exchanged messages. Using Gaussian approximation of the exchanged messages, we perform a two-dimensional density evolution analysis for the noisy LDPC d…
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols (codeword) with desired target probability distribution. The set of the output codewords constitutes a codebook (or code) of a DM. Constant-composition DM (CCDM) uses arithmetic coding to efficiently encode data into codewords from a constant-composition (CC) codebook. The CC constraint limits the size of the codebook, and hence the coding rate of the CCDM. The performance of CCDM degrades with decreasing output length. To improve the performance for short transmission blocks we present a class of multi…
It is well known that for Gaussian channels, a nearest neighbor decoding rule, which seeks the minimum Euclidean distance between a codeword and the received channel output vector, is the maximum likelihood solution and hence capacity-achieving. Nearest neighbor decoding remains a convenient and yet mismatched solution for general channels, and the key message of this paper is that the performance of the nearest neighbor decoding can be improved by generalizing its decoding metric to incorporate channel state dependent output processing and codeword scaling. Using generalized mutual informatio…
Short message noisy network coding (SNNC) differs from long message noisy network coding (LNNC) in that one transmits many short messages in blocks rather than using one long message with repetitive encoding. Several properties of SNNC are developed. First, SNNC with backward decoding achieves the same rates as SNNC with offset encoding and sliding window decoding for memoryless networks where each node transmits a multicast message. The rates are the same as LNNC with joint decoding. Second, SNNC enables early decoding if the channel quality happens to be good. This leads to mixed strategies …
The training complexity of deep learning-based channel decoders scales exponentially with the codebook size and therefore with the number of information bits. Thus, neural network decoding (NND) is currently only feasible for very short block lengths. In this work, we show that the conventional iterative decoding algorithm for polar codes can be enhanced when sub-blocks of the decoder are replaced by neural network (NN) based components. Thus, we partition the encoding graph into smaller sub-blocks and train them individually, closely approaching maximum a posteriori (MAP) performance per sub-…
We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users' codebooks, i.e., they perform oblivious processing. The destination, or central processor, however, is informed about the users' codebooks. We establish a single-letter characterization of the capacity region of this model for a class of discrete memoryless channels in which the outputs at the relay …
Low density parity check (LDPC) lattices are obtained from Construction D' and a family of nested binary LDPC codes. We consider an special case of these lattices with one binary LDPC code as underlying code. This special case of LDPC lattices can be obtained by lifting binary LDPC codes using Construction A lattices. The LDPC lattices were the first family of lattices which have efficient decoding in high dimensions. We employ the encoding and decoding of the LDPC lattices in a cooperative transmission framework. We establish two efficient shaping methods based on hypercube shaping and Vorono…
The sequence reconstruction problem, introduced by Levenshtein in 2001, considers a communication scenario where the sender transmits a codeword from some codebook and the receiver obtains multiple noisy reads of the codeword. The common setup assumes the codebook to be the entire space and the problem is to determine the minimum number of distinct reads that is required to reconstruct the transmitted codeword. Motivated by modern storage devices, we study a variant of the problem where the number of noisy reads $N$ is fixed. Specifically, we design reconstruction codes that reconstruct a code…
The asymptotic iterative decoding performances of low-density parity-check (LDPC) codes using min-sum (MS) and sum-product (SP) decoding algorithms on memoryless binary-input output-symmetric (MBIOS) channels are analyzed in this paper. For MS decoding, the analysis is done by upper bounding the bit error probability of the root bit of a tree code by the sequence error probability of a subcode of the tree code assuming the transmission of the all-zero codeword. The result is a recursive upper bound on the bit error probability after each iteration. For SP decoding, we derive a recursively dete…
We consider the problem of slotted asynchronous coded communication, where in each time frame (slot), the transmitter is either silent or transmits a codeword from a given (randomly selected) codebook. The task of the decoder is to decide whether transmission has taken place, and if so, to decode the message. We derive the optimum detection/decoding rule in the sense of the best trade-off among the probabilities of decoding error, false alarm, and misdetection. For this detection/decoding rule, we then derive single-letter characterizations of the exact exponential rates of these three probabi…
We develop a new transmission scheme for additive white Gaussian noisy (AWGN) channels based on Fuchsian groups from rational quaternion algebras. The structure of the proposed Fuchsian codes is nonlinear and nonuniform, hence conventional decoding methods based on linearity and symmetry do not apply. Previously, only brute force decoding methods with complexity that is linear in the code size exist for general nonuniform codes. However, the properly discontinuous character of the action of the Fuchsian groups on the complex upper half-plane translates into decoding complexity that is logarith…
Here we write in a unified fashion (using "R(P, Q, D)") the random coding exponents in channel coding and lossy source coding. We derive their explicit forms and show, that, for a given random codebook distribution Q, the channel decoding error exponent can be viewed as an encoding success exponent in lossy source coding, and the channel correct-decoding exponent can be viewed as an encoding failure exponent in lossy source coding. We then extend the channel exponents to arbitrary D, which corresponds for D > 0 to erasure decoding and for D < 0 to list decoding. For comparison, we also derive …
The transmission of digital data is one of the principal tasks in modern wireless communication. Classically, the communication channel consists of one transmitter and one receiver; however, due to the constantly increasing demand in higher transmission rates, the popularity of using several receivers and transmitters has been rapidly growing. In this paper, we combine a number of fairly standard techniques from numerical linear algebra and probability to develop several (apparently novel) randomized schemes for the decoding of digital messages sent over a noisy multivariate Gaussian channel. …
A modified Gradient Descent Bit Flipping (GDBF) algorithm is proposed for decoding Low Density Parity Check (LDPC) codes on the binary-input additive white Gaussian noise channel. The new algorithm, called Noisy GDBF (NGDBF), introduces a random perturbation into each symbol metric at each iteration. The noise perturbation allows the algorithm to escape from undesirable local maxima, resulting in improved performance. A combination of heuristic improvements to the algorithm are proposed and evaluated. When the proposed heuristics are applied, NGDBF performs better than any previously reported …
We prove two results on the universality of polar codes for source coding and channel communication. First, we show that for any polar code built for a source $P_{X,Z}$ there exists a slightly modified polar code - having the same rate, the same encoding and decoding complexity and the same error rate - that is universal for every source $P_{X,Y}$ when using successive cancellation decoding, at least when the channel $P_{Y|X}$ is more capable than $P_{Z|X}$ and $P_X$ is such that it maximizes $I(X;Y) - I(X;Z)$ for the given channels $P_{Y|X}$ and $P_{Z|X}$. This result extends to channel codin…
We consider the problem of erasure/list decoding using certain classes of simplified decoders. Specifically, we assume a class of erasure/list decoders, such that a codeword is in the list if its likelihood is larger than a threshold. This class of decoders both approximates the optimal decoder of Forney, and also includes the following simplified subclasses of decoding rules: The first is a function of the output vector only, but not the codebook (which is most suitable for high rates), and the second is a scaled version of the maximum likelihood decoder (which is most suitable for low rates)…
The problem of source coding with side information (SCSI) is closely related to channel coding. Therefore, existing literature focuses on using the most successful channel codes namely, LDPC codes, turbo codes, and their variants, to solve this problem assuming classical unique decoding of the underlying channel code. In this paper, in contrast to classical decoding, we have taken the list decoding approach. We show that syndrome source coding using list decoding can achieve the theoretical limit. We argue that, as opposed to channel coding, the correct sequence from the list produced by the l…
The goal of non-adaptive group testing is to identify at most $d$ defective items from $N$ items, in which a test of a subset of $N$ items is positive if it contains at least one defective item, and negative otherwise. However, in many cases, especially in biological screening, the outcome is unreliable due to biochemical interaction; i.e., \textit{noise.} Consequently, a positive result can change to a negative one (false negative) and vice versa (false positive). In this work, we first consider the dilution effect in which \textit{the degree of noise depends on the number of items in the tes…
We propose an irregular repetition slotted ALOHA (IRSA) based random-access protocol for the binary adder channel (BAC). The BAC captures important physical-layer concepts, such as packet generation, per-slot decoding, and information rate, which are neglected in the commonly considered collision channel model. We divide a frame into slots and let users generate a packet, to be transmitted over a slot, from a given codebook. In a state-of-the-art scheme proposed by Paolini et al. (2022), the codebook is constructed as the parity-check matrix of a BCH code. Here, we construct the codebook from …
A noisy network coding scheme for sending multiple sources over a general noisy network is presented. For multi-source multicast networks, the scheme naturally extends both network coding over noiseless networks by Ahlswede, Cai, Li, and Yeung, and compress-forward coding for the relay channel by Cover and El Gamal to general discrete memoryless and Gaussian networks. The scheme also recovers as special cases the results on coding for wireless relay networks and deterministic networks by Avestimehr, Diggavi, and Tse, and coding for wireless erasure networks by Dana, Gowaikar, Palanki, Hassibi,…
Decoding via sequentially guessing the error pattern in a received noisy sequence has received attention recently, and ORBGRAND has been proposed as one such decoding algorithm that is capable of utilizing the soft information embedded in the received noisy sequence. An information theoretic study is conducted for ORBGRAND, and it is shown that the achievable rate of ORBGRAND using independent and identically distributed random codebooks almost coincides with the channel capacity, for an additive white Gaussian noise channel under antipodal input. For finite-length codes, improved guessing sch…
This paper considers the problem of covert communication with mismatched decoding, in which a sender wishes to reliably communicate with a receiver whose decoder is fixed and possibly sub-optimal, and simultaneously to ensure that the communication is covert with respect to a warden. We present single-letter lower and upper bounds on the information-theoretically optimal throughput as a function of the given decoding metric, channel laws, and the desired level of covertness. These bounds match for a variety of scenarios of interest, such as (i) when the channel between the sender and receiver …
The goal of combining beamforming and space-time coding in this work is to obtain full-diversity order and to provide additional received power (array gain) compared to conventional space-time codes. In our system, we consider a quasi-static fading environment and we incorporate both high-rate and low-rate feedback channels with possible feedback errors. To utilize feedback information, a class of code constellations is proposed, inspired from orthogonal designs and precoded space-time block codes, which is called generalized partly orthogonal designs or generalized PODs. Furthermore, to model…
Departing from traditional communication theory where decoding algorithms are assumed to perform without error, a system where noise perturbs both computational devices and communication channels is considered here. This paper studies limits in processing noisy signals with noisy circuits by investigating the effect of noise on standard iterative decoders for low-density parity-check codes. Concentration of decoding performance around its average is shown to hold when noise is introduced into message-passing and local computation. Density evolution equations for simple faulty iterative decoder…
For information transmission a binary symmetric channel is used. There is also another noisy binary symmetric channel (feedback channel), and the transmitter observes without delay all the outputs of the forward channel via that feedback channel. The transmission of an exponential number of messages (i.e. the transmission rate is positive) is considered. The achievable decoding error exponent for such a combination of channels is investigated. It is shown that if the crossover probability of the feedback channel is less than a certain positive value, then the achievable error exponent is bette…
High-throughput and quantitative experimental technologies are experiencing rapid advances in the biological sciences. One important recent technique is multiplexed fluorescence in situ hybridization (mFISH), which enables the identification and localization of large numbers of individual strands of RNA within single cells. Core to that technology is a coding problem: with each RNA sequence of interest being a codeword, how to design a codebook of probes, and how to decode the resulting noisy measurements? Published work has relied on assumptions of uniformly distributed codewords and binary s…
School of Electronics and Information Technology, Sun Yat-sen University（中山大学电子与信息工程学院）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Hangzhou Research Center, Huawei Technologies Co., Ltd.（华为技术有限公司杭州研究中心）
