Over discrete memoryless channels (DMC), linear decoders (maximizing additive metrics) afford several nice properties. In particular, if suitable encoders are employed, the use of decoding algorithm with manageable complexities is permitted. Maximum likelihood is an example of linear decoder. For a compound DMC, decoders that perform well without the channel's knowledge are required in order to achieve capacity. Several such decoders have been studied in the literature. However, there is no such known decoder which is linear. Hence, the problem of finding linear decoders achieving capacity for compound DMC is addressed, and it is shown that under minor concessions, such decoders exist and can be constructed. This paper also develops a "local geometric analysis", which allows in particular, to solve the above problem. By considering very noisy channels, the original problem is reduced, in the limit, to an inner product space problem, for which insightful solutions can be found. The local setting can then provide counterexamples to disproof claims, but also, it is shown how in this problem, results proven locally can be "lifted" to results proven globally.
We consider the compound capacity of polar codes under successive cancellation decoding for a collection of binary-input memoryless output-symmetric channels. By deriving a sequence of upper and lower bounds, we show that in general the compound capacity under successive decoding is strictly smaller than the unrestricted compound capacity.
This work investigates the general two-user Compound Broadcast Channel (BC) where an encoder wishes to transmit common and private messages to two receivers while being oblivious to two possible channel realizations controlling the communication. The focus is on the characterization of the largest achievable rate region by resorting to more evolved encoding and decoding techniques than the conventional coding for the standard BC. The role of the decoder is first explored, and an achievable rate region is derived based on the principle of "Interference Decoding" (ID) where each receiver decodes…
We consider the problem of universal decoding for arbitrary unknown channels in the random coding regime. For a given random coding distribution and a given class of metric decoders, we propose a generic universal decoder whose average error probability is, within a sub-exponential multiplicative factor, no larger than that of the best decoder within this class of decoders. Since the optimum, maximum likelihood (ML) decoder of the underlying channel is not necessarily assumed to belong to the given class of decoders, this setting suggests a common generalized framework for: (i) mismatched deco…
As a new joint source-channel coding scheme, the double polar (D-Polar) codes have been proposed recently. In this letter, a novel joint source-channel decoder, namely the joint successive cancellation list (J-SCL) decoder, is proposed to improve the decoding performance of the D-Polar codes. We merge the trellis of the source polar code and that of the channel polar code to construct a compound trellis. In this compound trellis, the joint source-channel nodes represent both of the information bits and the high-entropy bits. Based on the compound trellis, the J-SCL decoder is designed to recov…
A universal decoding procedure is proposed for the intersymbol interference (ISI) Gaussian channels. The universality of the proposed decoder is in the sense of being independent of the various channel parameters, and at the same time, attaining the same random coding error exponent as the optimal maximum-likelihood (ML) decoder, which utilizes full knowledge of these unknown parameters. The proposed decoding rule can be regarded as a frequency domain version of the universal maximum mutual information (MMI) decoder. Contrary to previously suggested universal decoders for ISI channels, our pro…
A new straightforward universal blind detection algorithm for linear Gaussian channel with ISI is given. A new error exponent is derived, which is better than Gallager's random coding error exponent.
We consider a setting of Slepian--Wolf coding, where the random bin of the source vector undergoes channel coding, and then decoded at the receiver, based on additional side information, correlated to the source. For a given distribution of the randomly selected channel codewords, we propose a universal decoder that depends on the statistics of neither the correlated sources nor the channel, assuming first that they are both memoryless. Exact analysis of the random-binning/random-coding error exponent of this universal decoder shows that it is the same as the one achieved by the optimal maximu…
Polar coding over a class of binary discrete memoryless channels with channel knowledge at the encoder is studied. It is shown that polar codes achieve the capacity of convex and one-sided classes of symmetric channels.
A list decoding scheme for universal polar codes is presented. Our scheme applies to the universal polar codes first introduced by Sasoglu and Wang, and generalized to processes with memory by the authors. These codes are based on the concatenation of different polar transforms: a sequence of "slow" transforms and Arikan's original "fast" transform. List decoding of polar codes has been previously presented in the context of the fast transform. However, the slow transform is markedly different and requires new techniques and data structures. We show that list decoding is possible with space co…
Currently for the nt x nr MIMO channel, any explicitly constructed space-time (ST) designs that achieve optimality with respect to the diversity multiplexing tradeoff (DMT) are known to do so only when decoded using maximum likelihood (ML) decoding, which may incur prohibitive decoding complexity. In this paper we prove that MMSE regularized lattice decoding, as well as the computationally efficient lattice reduction (LR) aided MMSE decoder, allows for efficient and DMT optimal decoding of any approximately universal lattice-based code. The result identifies for the first time an explicitly co…
In this paper, we consider a few iterative decoding schemes for the joint source-channel coding of correlated sources. Specifically, we consider the joint source-channel coding of two erasure correlated sources with transmission over different erasure channels. Our main interest is in determining whether or not various code ensembles can achieve the capacity region universally over varying channel conditions. We consider two ensembles in the class of low-density generator-matrix (LDGM) codes known as Luby-Transform (LT) codes and one ensemble of low-density parity-check (LDPC) codes. We analyz…
Motivated by applications of rateless coding, decision feedback, and ARQ, we study the problem of universal decoding for unknown channels, in the presence of an erasure option. Specifically, we harness the competitive minimax methodology developed in earlier studies, in order to derive a universal version of Forney's classical erasure/list decoder, which in the erasure case, optimally trades off between the probability of erasure and the probability of undetected error. The proposed universal erasure decoder guarantees universal achievability of a certain fraction $ξ$ of the optimum error expo…
We consider the topic of universal decoding with a decoder that does not have direct access to the codebook, but only to noisy versions of the various randomly generated codewords, a problem motivated by biometrical identification systems. Both the source that generates the original (clean) codewords, and the channel that corrupts them in generating the noisy codewords, as well as the main channel for communicating the messages, are all modeled by non-unifilar, finite-state systems (hidden Markov models). As in previous works on universal decoding, here too, the average error probability of ou…
We derive universal codes for transmission of broadcast and confidential messages over classical-quantum-quantum and fully quantum channels. These codes are robust to channel uncertainties considered in the compound model. To construct these codes we generalize random codes for transmission of public messages, to derive a universal superposition coding for the compound quantum broadcast channel. As an application, we give a multi-letter characterization of regions corresponding to the capacity of the compound quantum broadcast channel for transmitting broadcast and confidential messages simult…
We introduce the two user finite state compound Gaussian interference channel and characterize its capacity region to within one bit. The main contributions involve both novel inner and outer bounds. The inner bound is multilevel superposition coding, but the decoding of the levels is opportunistic, depending on the channel state. The genie aided outer bound is motivated by the typical error events of the achievable scheme.
Motivated by applications of biometric identification and content identification systems, we consider the problem of random coding for channels, where each codeword undergoes lossy compression (vector quantization), and where the decoder bases its decision only on the compressed codewords and the channel output, which is in turn, the channel's response to the transmission of an original codeword, before compression. For memoryless sources and memoryless channels with finite alphabets, we propose a new universal decoder and analyze its error exponent, which improves on an earlier result by Dasa…
A new achievability scheme for the compound channel with discrete memoryless (DM) state noncausally available at the encoder is established. Achievability is proved using superposition coding, Marton coding, joint typicality encoding, and indirect decoding. The scheme is shown to achieve strictly higher rate than the straightforward extension of the Gelfand-Pinsker coding scheme for a single DMC with DM state, and is optimal for some classes of channels.
We consider the problem of (almost) lossless source coding of two correlated memoryless sources using separate encoders and a joint decoder, that is, Slepian-Wolf (S-W) coding. In our setting, the encoding and decoding are asynchronous, i.e., there is a certain relative delay between the two sources. Neither the source parameters nor the relative delay are known to the encoders and the decoder. Since we assume that both encoders implement standard random binning, which does not require such knowledge anyway, the focus of this work is on the decoder. Our main contribution is in proposing a univ…
We consider the multi-user lossy source-coding problem for continuous alphabet sources. In a previous work, Ziv proposed a single-user universal coding scheme which uses uniform quantization with dither, followed by a lossless source encoder (entropy coder). In this paper, we generalize Ziv's scheme to the multi-user setting. For this generalized universal scheme, upper bounds are derived on the redundancies, defined as the differences between the actual rates and the closest corresponding rates on the boundary of the rate region. It is shown that this scheme can achieve redundancies of no mor…
Universally achievable error exponents pertaining to certain families of channels (most notably, discrete memoryless channels (DMC's)), and various ensembles of random codes, are studied by combining the competitive minimax approach, proposed by Feder and Merhav, with Chernoff bound and Gallager's techniques for the analysis of error exponents. In particular, we derive a single--letter expression for the largest, universally achievable fraction $ξ$ of the optimum error exponent pertaining to the optimum ML decoding. Moreover, a simpler single--letter expression for a lower bound to $ξ$ is pres…
Random jammers that overpower transmitted signals are a practical concern for many wireless communication protocols. As such, wireless receivers must be able to cope with standard channel noise and jamming (intentional or unintentional). To address this challenge, we propose a novel method to augment the resilience of the recent family of universal error-correcting GRAND algorithms. This method, called Erasure Decoding by Gaussian Elimination (EDGE), impacts the syndrome check block and is applicable to any variant of GRAND. We show that the proposed EDGE method naturally reverts to the origin…
Sparse superposition codes, or sparse regression codes, constitute a new class of codes which was first introduced for communication over the additive white Gaussian noise (AWGN) channel. It has been shown that such codes are capacity-achieving over the AWGN channel under optimal maximum-likelihood decoding as well as under various efficient iterative decoding schemes equipped with power allocation or spatially coupled constructions. Here, we generalize the analysis of these codes to a much broader setting that includes all memoryless channels. We show, for a large class of memoryless channels…
We consider the compound memoryless quantum multiple-access channel (QMAC) with two sending terminals. In this model, the transmission is governed by the memoryless extensions of a completely positive and trace preserving map which can be any element of a prescribed set of possible maps. We study a communication scenario, where one of the senders aims for transmission of classical messages while the other sender sends quantum information. Combining powerful universal random coding results for classical and quantum information transmission over point-to-point channels, we establish universal co…
This paper is focused on the derivation of some universal properties of capacity-approaching low-density parity-check (LDPC) code ensembles whose transmission takes place over memoryless binary-input output-symmetric (MBIOS) channels. Properties of the degree distributions, graphical complexity and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information-theoretic bounds. These bounds are expressed in terms of the target block/ bit error probability and the gap (in rate) to capacity. Most of the bounds are general for any decoding algorithm, and…
We prove two results on the universality of polar codes for source coding and channel communication. First, we show that for any polar code built for a source $P_{X,Z}$ there exists a slightly modified polar code - having the same rate, the same encoding and decoding complexity and the same error rate - that is universal for every source $P_{X,Y}$ when using successive cancellation decoding, at least when the channel $P_{Y|X}$ is more capable than $P_{Z|X}$ and $P_X$ is such that it maximizes $I(X;Y) - I(X;Z)$ for the given channels $P_{Y|X}$ and $P_{Z|X}$. This result extends to channel codin…
We derive a lower bound on the secrecy capacity of the compound wiretap channel with channel state information at the transmitter which matches the general upper bound on the secrecy capacity of general compound wiretap channels given by Liang et al. and thus establishing a full coding theorem in this case. We achieve this with a quite strong secrecy criterion and with a decoder that is robust against the effect of randomisation in the encoding. This relieves us from the need of decoding the randomisation parameter which is in general not possible within this model. Moreover we prove a lower b…
We consider explicit constructions of multi-level lattice codes that universally approach the capacity of the compound block-fading channel. Specifically, building on algebraic partitions of lattices, we show how to construct codes with negligible probability of error for any channel realization and normalized log-density approaching the Poltyrev limit. Capacity analyses and numerical results on the achievable rates for each partition level are provided. The proposed codes have several enjoyable properties such as constructiveness and good decoding complexity, as compared to random one-level c…
Due to the short and bursty incoming messages, channel access activities in a wireless random access system are often fractional. The lack of frequent data support consequently makes it difficult for the receiver to estimate and track the time varying channel states with high precision. This paper investigates random multiple access communication over a compound wireless channel where channel realization is known neither at the transmitters nor at the receiver. An achievable rate and error probability tradeoff bound is derived under the non-asymptotic assumption of a finite codeword length. Th…
When information is to be transmitted over an unknown, possibly unreliable channel, an erasure option at the decoder is desirable. Using constant-composition random codes, we propose a generalization of Csiszar and Korner's Maximum Mutual Information decoder with erasure option for discrete memoryless channels. The new decoder is parameterized by a weighting function that is designed to optimize the fundamental tradeoff between undetected-error and erasure exponents for a compound class of channels. The class of weighting functions may be further enlarged to optimize a similar tradeoff for lis…
This work is on fast encoding and decoding of polar codes. We propose and detail 8-bit and 16-bit parallel decoders that can be used to reduce the decoding latency of the successive-cancellation decoder. These decoders are universal and can decode flexible-rate and flexible-length polar codes. We also present fast encoders that can be used to increase the throughput of serially-implemented polar encoders.
Consider the following framework of universal decoding suggested in [MerhavUniversal]. Given a family of decoding metrics and random coding distribution (prior), a single, universal, decoder is optimal if for any possible channel the average error probability when using this decoder is better than the error probability attained by the best decoder in the family up to a subexponential multiplicative factor. We describe a general universal decoder in this framework. The penalty for using this universal decoder is computed. The universal metric is constructed as follows. For each metric, a canoni…
Over binary input channels, uniform distribution is a universal prior, in the sense that it allows to maximize the worst case mutual information over all binary input channels, ensuring at least 94.2% of the capacity. In this paper, we address a similar question, but with respect to a universal generalized linear decoder. We look for the best collection of finitely many a posteriori metrics, to maximize the worst case mismatched mutual information achieved by decoding with these metrics (instead of an optimal decoder such as the Maximum Likelihood (ML) tuned to the true channel). It is shown t…
We consider wiretap channels with uncertainty on the eavesdropper channel under (i) noisy blockwise type II, (ii) compound, or (iii) arbitrarily varying models. We present explicit wiretap codes that can handle these models in a unified manner and only rely on three primitives, namely source coding with side information, universal hashing, and distribution approximation. Our explicit wiretap codes achieve the best known single-letter achievable rates, previously obtained non-constructively, for the models considered. Our results are obtained for strong secrecy, do not require a pre-shared secr…
We determine the capacity of compound classical-quantum channels. As a consequence we obtain the capacity formula for the averaged classical-quantum channels. The capacity result for compound channels demonstrates, as in the classical setting, the existence of reliable universal classical-quantum codes in scenarios where the only a priori information about the channel used for the transmission of information is that it belongs to a given set of memoryless classical-quantum channels. Our approach is based on the universal classical approximation of the quantum relative entropy which in turn rel…
We derive universal classical-quantum superposition coding and universal classical-quantum multiple access channel code by using generalized packing lemmas for the type method. Using our classical-quantum universal superposition code, we establish the capacity region of a classical-quantum compound broadcast channel with degraded message sets. Our universal classical-quantum multiple access channel codes have two types of codes. One is a code with joint decoding and the other is a code with separate decoding. The former universally achieves corner points of the capacity region and the latter u…
We present new efficient recursive decoders for the Barnes-Wall lattices based on their squaring construction. The analysis of the new decoders reveals a quasi-quadratic complexity in the lattice dimension and a quasi-linear complexity in the list-size. The error rate is shown to be close to the universal lower bound in dimensions 64 and 128.
We derive a lower bound on the secrecy capacity of the compound wiretap channel with channel state information at the transmitter which matches the general upper bound on the secrecy capacity of general compound wiretap channels given by Liang et al. and thus establishing a full coding theorem in this case. We achieve this with a stronger secrecy criterion and the maximum error probability criterion, and with a decoder that is robust against the effect of randomisation in the encoding. This relieves us from the need of decoding the randomisation parameter which is in general not possible withi…
We consider compound multi-input multi-output (MIMO) wiretap channels where minimal channel state information at the transmitter (CSIT) is assumed. Code construction is given for the special case of isotropic mutual information, which serves as a conservative strategy for general cases. Using the flatness factor for MIMO channels, we propose lattice codes universally achieving the secrecy capacity of compound MIMO wiretap channels up to a constant gap (measured in nats) that is equal to the number of transmit antennas. The proposed approach improves upon existing works on secrecy coding for MI…
An Optimal, Universal and Agnostic Decoding Method for Message Reconstruction, Bio and Technosignature Detection
School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）；The Alan Turing Institute（艾伦·图灵研究所）；Oxford Immune Algorithmics（牛津免疫算法公司）；The Arrival Institute（Arrival研究所）；King’s Institute for Artificial Intelligence, King’s College London（伦敦国王学院国王人工智能研究所）；Centre for Logic, Epistemology and the History of Science, University of Campinas（坎皮纳斯大学逻辑、认识论与科学史中心）；National Laboratory for Scientific Computing（国家科学计算实验室）
We put forth new models for universal channel coding. Unlike standard codes which are designed for a specific type of channel, our most general universal code makes communication resilient on every channel, provided the noise level is below the tolerated bound, where the noise level t of a channel is the logarithm of its ambiguity (the maximum number of strings that can be distorted into a given one). The other more restricted universal codes still work for large classes of natural channels. In a universal code, encoding is channel-independent, but the decoding function knows the type of chann…
Polar codes, invented by Arikan in 2009, are known to achieve the capacity of any binary-input memoryless output-symmetric channel. One of the few drawbacks of the original polar code construction is that it is not universal. This means that the code has to be tailored to the channel if we want to transmit close to capacity. We present two "polar-like" schemes which are capable of achieving the compound capacity of the whole class of binary-input memoryless output-symmetric channels with low complexity. Roughly speaking, for the first scheme we stack up $N$ polar blocks of length $N$ on top of…
We consider a decoder with an erasure option and a variable size list decoder for channels with non-casual side information at the transmitter. First, universally achievable error exponents are offered for decoding with an erasure option using a parameterized decoder in the spirit of Csiszár and Körner's decoder. Then, the proposed decoding rule is generalized by extending the range of its parameters to allow variable size list decoding. This extension gives a unified treatment for erasure/list decoding. Exponential bounds on the probability of list error and the average number of incorrect me…
We describe and analyze the joint source/channel coding properties of a class of sparse graphical codes based on compounding a low-density generator matrix (LDGM) code with a low-density parity check (LDPC) code. Our first pair of theorems establish that there exist codes from this ensemble, with all degrees remaining bounded independently of block length, that are simultaneously optimal as both source and channel codes when encoding and decoding are performed optimally. More precisely, in the context of lossy compression, we prove that finite degree constructions can achieve any pair $(R, D)$…
Guessing Random Additive Noise Decoding (GRAND) is a universal decoding algorithm that can be used to perform maximum likelihood decoding. It attempts to find the errors introduced by the channel by generating a sequence of possible error vectors in order of likelihood of occurrence and applying them to the received vector. Ordered reliability bits GRAND (ORBGRAND) integrates soft information received from the channel to refine the error vector sequence. In this work, ORBGRAND is modified to produce a soft output, to enable its use as an iterative soft-input soft-output (SISO) decoder. Three t…
