We derive a single-letter upper bound to the mismatched-decoding capacity for discrete memoryless channels. The bound is expressed as the mutual information of a transformation of the channel, such that a maximum-likelihood decoding error on the translated channel implies a mismatched-decoding error in the original channel. In particular, a strong converse is shown to hold for this upper-bound: if the rate exceeds the upper-bound, the probability of error tends to 1 exponentially when the block-length tends to infinity. We also show that the underlying optimization problem is a convex-concave problem and that an efficient iterative algorithm converges to the optimal solution. In addition, we show that, unlike achievable rates in the literature, the multiletter version of the bound does not improve. A number of examples are discussed throughout the paper.
Shaping gain is attained in schemes where a shaped subcode is chosen from a larger codebook by a codeword selection process. This includes the popular method of Trellis Shaping (TS), originally proposed by Forney for average power reduction. The decoding process of such schemes is mismatched, since it is aware of only the large codebook. This study models such schemes by a random code construction and derives achievable bounds on the transmission rate under matched and mismatched decoding. For matched decoding the bound is obtained using a modified asymptotic equipartition property (AEP) theor…
We show that the mismatched capacity of binary discrete memoryless channels can be improved by channel combining and splitting via Arıkan's polar transformations. We also show that the improvement is possible even if the transformed channels are decoded with a mismatched polar decoder.
This paper studies channel coding for the discrete memoryless multiple-access channel with a given (possibly suboptimal) decoding rule. A multi-letter successive decoding rule depending on an arbitrary non-negative decoding metric is considered, and achievable rate regions and error exponents are derived both for the standard MAC (independent codebooks), and for the cognitive MAC (one user knows both messages) with superposition coding. In the cognitive case, the rate region and error exponent are shown to be tight with respect to the ensemble average. The rate regions are compared with those …
The fundamental limits of channels with mismatched decoding are addressed. A general formula is established for the mismatch capacity of a general channel, defined as a sequence of conditional distributions with a general decoding metrics sequence. We deduce an identity between the Verdú-Han general channel capacity formula, and the mismatch capacity formula applied to Maximum Likelihood decoding metric. Further, several upper bounds on the capacity are provided, and a simpler expression for a lower bound is derived for the case of a non-negative decoding metric. The general formula is special…
Arıkan's polar coding, is by now a well studied technique that allows achieving the symmetric capacity of binary input memoryless channels with low complexity encoding and decoding, provided that the polar decoding architecture is used and the decoding metric is matched to the true channel. In this paper, we analyze communication rates that are achievable when the polar coding/decoding architecture is used with the decoder using an incorrect model of the channel. We define the `polar mismatched capacity' as an analogue of the classical mismatched capacity, give an expression for it, and derive…
We introduce a new analysis technique to derive a single-letter upper bound on the mismatch capacity of a stationary, single-user, memoryless channel with a decoding metric $q$. Our bound is obtained by considering a multicast transmission over a two-user broadcast channel with decoding metrics $q$ and $ρ$ at the receivers, referred to as $(q,ρ)$-surely degraded. This channel has the property that the intersection event of correct $q$-decoding of receiver $1$ and erroneous $ρ$-decoding of receiver $2$ has zero probability for any fixed-composition codebook of a certain composition $P$. Our bou…
We develop a novel framework for proving converse theorems for channel coding, which is based on the analysis technique of multicast transmission with an additional auxiliary receiver, which serves as a genie to the original receiver. The genie provides the original receiver a certain narrowed list of codewords to choose from that includes the transmitted one. This technique is used to derive upper bounds on the mismatch capacity of discrete memoryless channels as well as the reliability function with a mismatched decoding metric. Unlike previous works, our bounding technique exploits also the…
For the class of mixed channels decomposed into stationary memoryless channels, single-letter characterizations of the $\varepsilon$-capacity have not been known except for restricted classes of channels such as the regular decomposable channel introduced by Winkelbauer. This paper gives single-letter characterizations of $\varepsilon$-capacity for mixed channels decomposed into at most countably many memoryless channels with a finite input alphabet and a general output alphabet with/without cost constraints. It is shown that a given characterization reduces to the one for the channel capacity…
We study finite alphabet channels with Unit Memory on the previous Channel Outputs called UMCO channels. We identify necessary and sufficient conditions, to test whether the capacity achieving channel input distributions with feedback are time-invariant, and whether feedback capacity is characterized by single letter, expressions, similar to that of memoryless channels. The method is based on showing that a certain dynamic programming equation, which in general, is a nested optimization problem over the sequence of channel input distributions, reduces to a non-nested optimization problem. More…
Over the past decades, the problem of communication over finite-state Markov channels (FSMCs) has been investigated in many researches and the capacity of FSMCs has been studied in closed form under the assumption of the availability of partial/complete channel state information at the sender and/or the receiver. In our work, we focus on infinite-state Markov channels by investigating the problem of message transmission over time-varying single-user multiple-input multiple-output (MIMO) Gauss-Markov Rayleigh fading channels with average power constraint and with complete channel state informat…
Regular perturbation is applied to the Manakov equation and motivates a generalized correlated phase-and-additive noise model for wavelength-division multiplexing over dual-polarization optical fiber channels. The model includes three hidden Gauss-Markov processes: phase noise, polarization rotation, and additive noise. Particle filtering is used to compute lower bounds on the capacity of multi-carrier communication with frequency-dependent powers and delays. A gain of 0.17 bits/s/Hz/pol in spectral efficiency or 0.8 dB in power efficiency is achieved with respect to existing models at their p…
This paper considers the problem of channel coding over Gaussian intersymbol interference (ISI) channels with a given metric decoding rule. Specifically, it is assumed that the mismatched decoder has an incorrect assumption on the impulse response function. The mismatch capacity is the highest achievable rate for a given decoding rule. Existing lower bounds to the mismatch capacity for channels and decoding metrics with memory (as in our model) are presented only in the form of multi-letter expressions that have not been calculated in practice. Consequently, they provide little insight on the …
A correlated phase-and-additive-noise (CPAN) mismatched model is developed for wavelength division multiplexing over optical fiber channels governed by the nonlinear Schrödinger equation. Both the phase and additive noise processes of the CPAN model are Gauss-Markov whereas previous work uses Wiener phase noise and white additive noise. Second order statistics are derived and lower bounds on the capacity are computed by simulations. The CPAN model characterizes nonlinearities better than existing models in the sense that it achieves better information rates. For example, the model gains 0.35 d…
The problem of mismatched decoding for discrete memoryless channels is addressed. A mismatched cognitive multiple-access channel is introduced, and an inner bound on its capacity region is derived using two alternative encoding methods: superposition coding and random binning. The inner bounds are derived by analyzing the average error probability of the code ensemble for both methods and by a tight characterization of the resulting error exponents. Random coding converse theorems are also derived. A comparison of the achievable regions shows that in the matched case, random binning performs a…
We derive a sphere-packing error exponent for coded transmission over discrete memoryless channels with a fixed decoding metric. By studying the error probability of the code over an auxiliary channel, we find a lower bound to the probability of error of mismatched decoding. The bound is shown to decay exponentially for coding rates smaller than a new upper bound to the mismatch capacity which is established in this paper. For rates higher than the new upper bound, the error probability is shown to be bounded away from zero. The new upper bound is shown to improve over previous upper bounds to…
An upper bound on the feedback capacity of unifilar finite-state channels (FSCs) is derived. A new technique, called the $Q$-contexts, is based on a construction of a directed graph that is used to quantize recursively the receiver's output sequences to a finite set of contexts. For any choice of $Q$-graph, the feedback capacity is bounded by a single-letter expression, $C_\text{fb}\leq \sup I(X,S;Y|Q)$, where the supremum is over $P_{X|S,Q}$ and the distribution of $(S,Q)$ is their stationary distribution. It is shown that the bound is tight for all unifilar FSCs where feedback capacity is kn…
The capacity of finite state channels (FSCs) with feedback has been shown to be a limit of a sequence of multi-letter expressions. Despite many efforts, a closed-form single-letter capacity characterization is unknown to date. In this paper, the feedback capacity is studied from a fundamental algorithmic point of view by addressing the question of whether or not the capacity can be algorithmically computed. To this aim, the concept of Turing machines is used, which provides fundamental performance limits of digital computers. It is shown that the feedback capacity of FSCs is not Banach-Mazur c…
We consider the first-order theory of random variables with the probabilistic independence relation, which concerns statements consisting of random variables, the probabilistic independence symbol, logical operators, and existential and universal quantifiers. Although probabilistic independence is the only non-logical relation included, this theory is surprisingly expressive, and is able to interpret the true first-order arithmetic over natural numbers (and hence is undecidable). We also construct a single-letter characterization of the capacity region for a general class of multiuser coding s…
We extend the notion of list decoding to {\em ratio list decoding} which involves a list decoder whose list size is specified as a function of the number of messages $M_n$ and the block length $n$. We present necessary and sufficient conditions on $M_n$ for the existence of code sequences which enable reliable list decoding with respect to the desired list size $L(M_n,n)$. It is shown that the ratio-capacity, defined as the supremum of achievable normalized logarithms of the ratio $r(M_n,n)=M_n/L(M_n,n)$ is equal to the Shannon channel capacity $C$, for both stochastic and deterministic encodi…
We revisit the information-theoretic analysis of bit-interleaved coded modulation (BICM) by modeling the BICM decoder as a mismatched decoder. The mismatched decoding model is well-defined for finite, yet arbitrary, block lengths, and naturally captures the channel memory among the bits belonging to the same symbol. We give two independent proofs of the achievability of the BICM capacity calculated by Caire et al. where BICM was modeled as a set of independent parallel binary-input channels whose output is the bitwise log-likelihood ratio. Our first achievability proof uses typical sequences, …
Spectral Approximation and Ergodic-Capacity Convergence of HMIMO Channels under Spatial-Wavenumber Domain Mismatch
Hangzhou Institute of Technology, Xidian University（西安电子科技大学杭州研究院）；Department of Mathematics and Algorithms Research, Nokia Bell Labs（诺基亚贝尔实验室数学与算法研究部）；School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与工程学院）
The common approach of designing a communication device is to maximize a well-defined objective function, e.g., the channel capacity and the cut-off rate. We propose easy-to-implement solutions for Gaussian channels that approximate the optimal results for these maximization problems. Three topics are addressed. First, we consider the case where the channel output is quantized, and we find the quantization thresholds that maximize the mutual information. The approximation derived from the asymptotic solution has a negligible loss on the entire range of SNR when 2-PAM modulation is used, and it…
We consider an ensemble of constant composition codes that are subsets of linear codes: while the encoder uses only the constant-composition subcode, the decoder operates as if the full linear code was used, with the motivation of simultaneously benefiting both from the probabilistic shaping of the channel input and from the linear structure of the code. We prove that the codebook mismatch can be fully compensated by using a mismatched additive decoding metric that achieves the random coding error exponent of (non-linear) constant composition codes. As the coding rate tends to the mutual infor…
We provide a rigorous definition of a certain kind of characterization for capacity regions of a family of Markoff networks which is based on optimization problems resulting out of calculating conditional mutual information from a finite number of random variables and by constraining these random variables in a certain way. This definition is partly motivated by the definition of single-letter characterizations in information theory. For a point-to-point Markoff channel, we prove that approximating the solution to these characterizations within an additive constant is a computable problem. Bas…
School of Electrical and Computer Engineering, Ben-Gurion University of the Negev（内盖夫本-古里安大学电气与计算机工程学院）；Department of Electrical and Computer Engineering, McMaster University（麦马斯特大学电气与计算机工程系）
This paper considers the problem of covert communication with mismatched decoding, in which a sender wishes to reliably communicate with a receiver whose decoder is fixed and possibly sub-optimal, and simultaneously to ensure that the communication is covert with respect to a warden. We present single-letter lower and upper bounds on the information-theoretically optimal throughput as a function of the given decoding metric, channel laws, and the desired level of covertness. These bounds match for a variety of scenarios of interest, such as (i) when the channel between the sender and receiver …
We consider the problem of reliable communication over multiple-access channels (MAC) where the channel is driven by an independent and identically distributed state process and the encoders and the decoder are provided with various degrees of asymmetric noisy channel state information (CSI). For the case where the encoders observe causal, asymmetric noisy CSI and the decoder observes complete CSI, we provide inner and outer bounds to the capacity region, which are tight for the sum-rate capacity. We then observe that, under a Markov assumption, similar capacity results also hold in the case w…
Motivated by recent developments in wireless power transfer, we study communication with a remotely powered transmitter. We propose an information-theoretic model where a charger can dynamically decide on how much power to transfer to the transmitter based on its side information regarding the communication, while the transmitter needs to dynamically adapt its coding strategy to its instantaneous energy state, which in turn depends on the actions previously taken by the charger. We characterize the capacity as an $n$-letter mutual information rate under various levels of side information avail…
The problem of mismatched decoding with an additive metric $q$ for a discrete memoryless channel $W$ is addressed. The "product-space" improvement of the random coding lower bound on the mismatch capacity, $C_q^{(\infty)}(W)$, was introduced by Csiszár and Narayan. We study two kinds of decoders. The {\it $δ$-margin mismatched decoder} outputs a message whose metric with the channel output exceeds that of all the other codewords by at least $δ$. The {\it $τ$-threshold decoder} outputs a single message whose metric with the channel output exceeds a threshold $τ$. Both decoders declare an error …
The capacity of finite state channels (FSCs) has been established as the limit of a sequence of multi-letter expressions only and, despite tremendous effort, a corresponding finite-letter characterization remains unknown to date. This paper analyzes the capacity of FSCs from a fundamental, algorithmic point of view by studying whether or not the corresponding achievability and converse bounds on the capacity can be computed algorithmically. For this purpose, the concept of Turing machines is used which provide the fundamental performance limits of digital computers. To this end, computable con…
For general memoryless systems, the typical information theoretic solution - when exists - has a "single-letter" form. This reflects the fact that optimum performance can be approached by a random code (or a random binning scheme), generated using independent and identically distributed copies of some single-letter distribution. Is that the form of the solution of any (information theoretic) problem? In fact, some counter examples are known. The most famous is the "two help one" problem: Korner and Marton showed that if we want to decode the modulo-two sum of two binary sources from their inde…
The channel capacity theorem for additive white Gaussian noise channel (AWGN), widely known as the Shannon-Hartley Law, expresses the information capacity of a channel bandlimited in the conventional Fourier domain in terms of the signal-to-noise ratio in it. In this letter generalized versions of the Shannon-Hartley Law using the linear canonical transform (LCT) are presented. The channel capacity for AWGN channels is found to be a function of the LCT parameters.
A class of burst noise-erasure channels which incorporate both errors and erasures during transmission is studied. The channel, whose output is explicitly expressed in terms of its input and a stationary ergodic noise-erasure process, is shown to have a so-called "quasi-symmetry" property under certain invertibility conditions. As a result, it is proved that a uniformly distributed input process maximizes the channel's block mutual information, resulting in a closed-form formula for its non-feedback capacity in terms of the noise-erasure entropy rate and the entropy rate of an auxiliary erasur…
In this paper, we consider the problem of evaluating capacity expression of a multiple access channel (MAC) with noiseless feedback. So far, the capacity expression for this channel is known through a multi letter directed information by Kramer [1]. Recently, it was shown in [2] that one can pose it as a dynamic optimization problem, however, no dynamic program was provided as the authors claimed there is no notion of state that is observed by both the senders. In this paper, we build upon [2] to show that there indeed exists a state and therefore a dynamic program (DP) that decomposes this dy…
We consider the problem of signal estimation (denoising) from a statistical-mechanical perspective, in continuation to a recent work on the analysis of mean-square error (MSE) estimation using a direct relationship between optimum estimation and certain partition functions. The paper consists of essentially two parts. In the first part, using the aforementioned relationship, we derive single-letter expressions of the mismatched MSE of a codeword (from a randomly selected code), corrupted by a Gaussian vector channel. In the second part, we provide several examples to demonstrate phase transiti…
In the classical Secret-Key generation model, Common Randomness is generated by two terminals based on the observation of correlated components of a common source, while keeping it secret from a non-legitimate observer. It is assumed that the statistics of the source are known to all participants. In this work, the Secret-Key generation based on a compound source is studied where the realization of the source statistic is unknown. The protocol should guarantee the security and reliability of the generated Secret-Key, simultaneously for all possible realizations of the compound source. A single…
Fluid antenna represents a concept where a mechanically flexible antenna can switch its location freely within a given space. Recently, it has been reported that even with a tiny space, a single-antenna fluid antenna system (FAS) can outperform an L-antenna maximum ratio combining (MRC) system in terms of outage probability if the number of locations (or ports) the fluid antenna can be switched to, is large enough. This letter aims to study if extraordinary capacity can also be achieved by FAS with a small space. We do this by deriving the ergodic capacity, and a capacity lower bound. This let…
We consider the Gaussian channel with power constraint P. A gap exists between the channel capacity and the highest achievable rate of equiprobable uniformly spaced signal. Several approaches enable to overcome this limitation such as constellations with non-uniform probability or constellation shaping. In this letter, we focus on constellation shaping. We give a construction of amplitude and phase-shift keying (APSK) constellations with equiprobable signaling that achieve the Gaussian capacity as the number of constellation points goes to infinity.
A general formalization is given for asynchronous multiple access channels which admits different assumptions on delays. This general framework allows the analysis of so far unexplored models leading to new interesting capacity regions. In particular, a single letter characterization is given for the capacity region in case of 3 senders, 2 synchronous with each other and the third not synchronous with them.
Northern Arizona University（北亚利桑那大学）；University of Wisconsin-Eau Claire（威斯康星大学欧克莱尔分校）；Arizona State University（亚利桑那州立大学）；DEVCOM Army Research Laboratory（美国陆军研究开发中心陆军研究实验室）；New Jersey Institute of Technology（新泽西理工学院）
We provide a single-letter characterization for the capacity region of a class of discrete degraded interference channels (DDICs). The class of DDICs considered includes the discrete additive degraded interference channel (DADIC) studied by Benzel. We show that for the class of DDICs studied, encoder cooperation does not increase the capacity region, and therefore, the capacity region of the class of DDICs is the same as the capacity region of the corresponding degraded broadcast channel.
We consider the capacity of memoryless finite-state multiple access channel (FS-MAC) with causal asymmetric noisy state information available at both transmitters and complete state information available at the receiver. Single letter inner and outer bounds are provided for the capacity of such channels when the state process is independent and identically distributed. The outer bound is attained by observing that the proposed inner bound is tight for the sum-rate capacity.
We introduce a general framework for treating channels with memory and feedback. First, we generalize Massey's concept of directed information and use it to characterize the feedback capacity of general channels. Second, we present coding results for Markov channels. This requires determining appropriate sufficient statistics at the encoder and decoder. Third, a dynamic programming framework for computing the capacity of Markov channels is presented. Fourth, it is shown that the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity.…
Previous works established the capacity region for some special cases of discrete memoryless degraded cognitive interference channel (CIC) with unidirectional destination cooperation (UDC). In this letter, we characterize the capacity region of the general discrete memoryless degraded CIC-UDC. The obtained results imply that the capacity region is achieved by the Gel'fand-Pinsker coding at the cognitive transmitter, superposition coding at the primary transmitter and decode-and-forward at the relay. Furthermore, using this general result and a novel converse analysis, we establish the capacity…
We consider an energy harvesting multiple access channel (MAC) where the transmitters are powered by an exogenous stochastic energy harvesting process and equipped with finite batteries. We characterize the capacity region of this channel as n-letter mutual information rate and develop inner and outer bounds that differ by a constant gap. An interesting conclusion that emerges from our results is that the sum-capacity approaches that of a standard AWGN MAC (with only an average constraint on the transmitted power), as the number of users in the MAC becomes large.
The capacity region of the multiple access channel with arbitrarily correlated sources remains an open problem. Cover, El Gamal and Salehi gave an achievable region in the form of single-letter entropy and mutual information expressions, without a single-letter converse. Cover, El Gamal and Salehi also gave a converse in terms of some n-letter mutual informations, which are incomputable. In this paper, we derive an upper bound for the sum rate of this channel in a single-letter expression by using spectrum analysis. The incomputability of the sum rate of Cover, El Gamal and Salehi scheme comes…
A single-letter characterization is provided for the capacity region of finite-state multiple access channels. The channel state is a Markov process, the transmitters have access to delayed state information, and channel state information is available at the receiver. The delays of the channel state information are assumed to be asymmetric at the transmitters. We apply the result to obtain the capacity region for a finite-state Gaussian MAC, and for a finite-state multiple-access fading channel. We derive power control strategies that maximize the capacity region for these channels.
Consider a memoryless relay channel, where the relay is connected to the destination with an isolated bit pipe of capacity $C_0$. Let $C(C_0)$ denote the capacity of this channel as a function of $C_0$. What is the critical value of $C_0$ such that $C(C_0)$ first equals $C(\infty)$? This is a long-standing open problem posed by Cover and named "The Capacity of the Relay Channel," in $Open \ Problems \ in \ Communication \ and \ Computation$, Springer-Verlag, 1987. In this paper, we answer this question in the Gaussian case and show that $C(C_0)$ can not equal to $C(\infty)$ unless $C_0=\infty$…
We investigate the problem of message transmission over time-varying single-user multiple-input multiple-output (MIMO) Rayleigh fading channels with average power constraint and with complete channel state information available at the receiver side (CSIR). To describe the channel variations over the time, we consider a first-order Gauss-Markov model. We completely solve the problem by giving a single-letter characterization of the channel capacity in closed form and by providing a rigorous proof of it.
We consider a public multi-user information embedding (watermarking) system in which two messages (watermarks) are independently embedded into two correlated covertexts and are transmitted through a multiple-access attack channel. The tradeoff between the achievable embedding rates and the average distortions for the two embedders is studied. For given distortion levels, inner and outer bounds for the embedding capacity region are obtained in single-letter form. Tighter bounds are also given for independent covertexts.
An arbitrarily varying wiretap channel (AVWTC) with a type constraint on the allowed state sequences is considered, and a single-letter characterization of its correlated-random (CR) assisted semantic-security (SS) capacity is derived. The allowed state sequences are the ones in a typical set around a single constraining type. SS is established by showing that the mutual information between the message and the eavesdropper's observations is negligible even when maximized over all message distributions, choices of state sequences and realizations of the CR-code. Both the achievability and the c…
Consider a memoryless degraded broadcast channel (DBC) in which the channel output is a single-letter function of the channel input and the channel noise. As examples, for the Gaussian broadcast channel (BC) this single-letter function is regular Euclidian addition and for the binary-symmetric BC this single-letter function is Galois-Field-two addition. This paper identifies several classes of discrete memoryless DBCs for which a relatively simple encoding scheme, which we call natural encoding, achieves capacity. Natural Encoding (NE) combines symbols from independent codebooks (one for each …
