The essential interactive capacity of a discrete memoryless channel is defined in this paper as the maximal rate at which the transcript of any interactive protocol can be reliably simulated over the channel, using a deterministic coding scheme. In contrast to other interactive capacity definitions in the literature, this definition makes no assumptions on the order of speakers (which can be adaptive) and does not allow any use of private / public randomness; hence, the essential interactive capacity is a function of the channel model only. It is shown that the essential interactive capacity of any binary memoryless symmetric (BMS) channel is at least $0.0302$ its Shannon capacity. To that end, we present a simple coding scheme, based on extended-Hamming codes combined with error detection, that achieves the lower bound in the special case of the binary symmetric channel (BSC). We then adapt the scheme to the entire family of BMS channels, and show that it achieves the same lower bound using extremes of the Bhattacharyya parameter.
The sum capacity of a class of discrete memoryless interference channels is determined. This class of channels is defined analogous to the Gaussian Z-interference channel with weak interference; as a result, the sum capacity is achieved by letting the transceiver pair subject to the interference communicates at a rate such that its message can be decoded at the unintended receiver using single user detection. Moreover, this class of discrete memoryless interference channels is equivalent in capacity region to certain discrete degraded interference channels. This allows the construction of a ca…
The capacity of a time-varying block-memoryless channel in which the transmitter and the receiver have access to (possibly different) noisy causal channel side information (CSI) is obtained. It is shown that the capacity formula obtained in this correspondence reduces to the capacity formula reported in \cite{Gold07} for the special case where the transmitter CSI is a deterministic function of the receiver CSI.
This paper studies the capacity of a class of discrete memoryless interference channels where interference is defined analogous to that of Gaussian interference channel with one-sided weak interference. The sum-rate capacity of this class of channels is determined. As with the Gaussian case, the sum-rate capacity is achieved by letting the transceiver pair subject to interference communicate at a rate such that its message can be decoded at the unintended receiver using single user detection. It is also established that this class of discrete memoryless interference channels is equivalent in c…
For most discrete memoryless channels, there does not exist a linear code for the channel which uses all of the channel's input symbols. Therefore, linearity of the code for such channels is a very restrictive condition and there should be a loosening of the algebraic structure of the code to a degree that the code can admit any channel input alphabet. For any channel input alphabet size, there always exists an Abelian group structure defined on the alphabet. We investigate the capacity of Abelian group codes over discrete memoryless channels and provide lower and upper bounds on the capacity.
We consider the capacity of memoryless finite-state multiple access channel (FS-MAC) with causal asymmetric noisy state information available at both transmitters and complete state information available at the receiver. Single letter inner and outer bounds are provided for the capacity of such channels when the state process is independent and identically distributed. The outer bound is attained by observing that the proposed inner bound is tight for the sum-rate capacity.
For the class of mixed channels decomposed into stationary memoryless channels, single-letter characterizations of the $\varepsilon$-capacity have not been known except for restricted classes of channels such as the regular decomposable channel introduced by Winkelbauer. This paper gives single-letter characterizations of $\varepsilon$-capacity for mixed channels decomposed into at most countably many memoryless channels with a finite input alphabet and a general output alphabet with/without cost constraints. It is shown that a given characterization reduces to the one for the channel capacity…
We investigate properties of a channel coding scheme leading to the minimum-possible frame error ratio when transmitting over a memoryless channel with rate R>C. The results are compared to the well-known properties of a channel coding scheme leading to minimum bit error ratio. It is concluded that these two optimization requests are contradicting. A valuable application of the derived results is presented.
We propose a method to increase the capacity achieved by uniform prior in discrete memoryless channels (DMC) with high input cardinality. It consists in appropriately reducing the input set. Different design criteria of the input subset are discussed. We develop an efficient algorithm to solve this problem based on the maximization of the cut-off rate. The method is applied to a mono-bit transceiver MIMO system, and it is shown that the capacity can be approached within tenths of a dB by employing standard binary codes while avoiding the use of distribution shapers.
We address the capacity of a discrete-time memoryless Gaussian channel, where the channel state information (CSI) is neither available at the transmitter nor at the receiver. The optimal capacity-achieving input distribution at low signal-to-noise ratio (SNR) is precisely characterized, and the exact capacity of a non-coherent channel is derived. The derived relations allow to better understanding the capacity of non-coherent channels at low SNR. Then, we compute the non-coherence penalty and give a more precise characterization of the sub-linear term in SNR. Finally, in order to get more insi…
In this paper, we study a model of communication under adversarial noise. In this model, the adversary makes online decisions on whether to corrupt a transmitted bit based on only the value of that bit. Like the usual binary symmetric channel of information theory or the fully adversarial channel of combinatorial coding theory, the adversary can, with high probability, introduce at most a given fraction of error. It is shown that, the capacity (maximum rate of reliable information transfer) of such memoryless adversary is strictly below that of the binary symmetric channel. We give new upper b…
The memoryless noncoherent single-input single-output (SISO) Rayleigh-fading channel is considered. Closed-form expressions for the mutual information between the output and the input of this channel when the input magnitude distribution is discrete and restricted to having two mass points are derived, and it is subsequently shown how these expressions can be used to obtain closed-form expressions for the capacity of this channel for signal to noise ratio (SNR) values of up to approximately 0 dB, and a tight capacity lower bound for SNR values between 0 dB and 10 dB. The expressions for the ch…
We study the determination problem of the channel capacity for the discrete memoryless channels in the finite blocklength regime. We derive explicit lower and upper bounds of the capacity. We shall demonstrate that the information spectrum approach is quite useful for investigating this problem.
The capacity of a discrete-time memoryless channel, in which successive symbols fade independently, and where the channel state information (CSI) is neither available at the transmitter nor at the receiver, is considered at low SNR. We derive a closed form expression of the optimal capacity-achieving input distribution at low signal-to-noise ratio (SNR) and give the exact capacity of a non-coherent channel at low SNR. The derived relations allow to better understanding the capacity of non-coherent channels at low SNR and bring an analytical answer to the peculiar behavior of the optimal input …
A coding scheme for the discrete memoryless broadcast channel with {noiseless, noisy, generalized} feedback is proposed, and the associated achievable region derived. The scheme is based on a block-Markov strategy combining the Marton scheme and a lossy version of the Gray-Wyner scheme with side-information. In each block the transmitter sends fresh data and update information that allows the receivers to improve the channel outputs observed in the previous block. For a generalization of Dueck's broadcast channel our scheme achieves the noiseless-feedback capacity, which is strictly larger tha…
The capacity region of the 3-user Gaussian Interference Channel (GIC) with mixed strong-very strong interference was established in \cite{ChS}. The mixed strong-very strong interference conditions considered in \cite{ChS} correspond to the case where, at each receiver, one of the interfering signals is strong and the other is very strong. In this paper, we derive the capacity region of $K$-user $(K\geq 3)$ Discrete Memoryless Interference Channels (DMICs) with a mixed strong-very strong interference. This corresponds to the case where, at each receiver one of the interfering signals is strong …
A single-letter characterization is provided for the capacity region of finite-state multiple-access channels, when the channel state process is an independent and identically distributed sequence, the transmitters have access to partial (quantized) state information, and complete channel state information is available at the receiver. The partial channel state information is assumed to be asymmetric at the encoders. As a main contribution, a tight converse coding theorem is presented. The difficulties associated with the case when the channel state has memory are discussed and connections to …
In this report, we generalize Shannon's push-to-talk two-way channel (PTT-TWC) by allowing reliable full-duplex transmission as well as noisy reception in the half-duplex (PTT) mode. Viewing a PTT-TWC as two state-dependent one-way channels, we introduce a channel symmetry property pertaining to the one-way channels. Shannon's TWC capacity inner bound is shown to be tight for the generalized model under this symmetry property. We also analytically derive the capacity region, which is shown to be the convex hull of (at most) 4 rate pairs. Examples that illustrate different shapes of the capacit…
While capacities of discrete memoryless channels are well studied, it is still not possible to obtain a closed-form expression for the capacity of an arbitrary discrete memoryless channel. This paper describes an elementary technique based on Karush Kuhn Tucker (KKT) conditions to obtain (1) a good upper bound of a discrete memoryless channel having an invertible positive channel matrix and (2) a closed-form expression for the capacity if the channel matrix satisfies certain conditions related to its singular value and its Gershgorin disk.
It is shown that nested polar codes achieve the Shannon rate-distortion function for arbitrary (binary or non-binary) discrete memoryless sources and the Shannon capacity of arbitrary discrete memoryless channels.
Memoryless channels with synchronization errors as defined by a stochastic channel matrix allowing for symbol insertions and deletions in addition to random errors are considered. Such channels are information stable, hence their Shannon capacity exists. However, computation of the channel capacity is formidable, and only some upper and lower bounds on the capacity (for some special cases) exist. In this short paper, using a simple methodology, we prove that the channel capacity is a convex function of the stochastic channel matrix. Since the more widely studied model of an independent identic…
This paper shows that the logarithm of the epsilon-error capacity (average error probability) for n uses of a discrete memoryless channel is upper bounded by the normal approximation plus a third-order term that does not exceed 1/2 log n + O(1) if the epsilon-dispersion of the channel is positive. This matches a lower bound by Y. Polyanskiy (2010) for discrete memoryless channels with positive reverse dispersion. If the epsilon-dispersion vanishes, the logarithm of the epsilon-error capacity is upper bounded by the n times the capacity plus a constant term except for a small class of DMCs and …
For any channel with a convex constraint set and finite Augustin capacity, existence of a unique Augustin center and associated Erven-Harremoes bound are established. Augustin-Legendre capacity, center, and radius are introduced and proved to be equal to the corresponding Renyi-Gallager entities. Sphere packing bounds with polynomial prefactors are derived for codes on two families of channels: (possibly non-stationary) memoryless channels with multiple additive cost constraints and stationary memoryless channels with convex constraints on the empirical distribution of the input codewords.
In this work we study the capacity advantage achieved by adding a single bit of communication -- not a link of capacity 1 but a single bit over all time -- to a memoryless network. Specifically, we present a memoryless network in which adding a single bit of communication strictly increases the capacity region.
A memoryless state-dependent broadcast channel (BC) is considered, where the transmitter wishes to convey two private messages to two receivers while simultaneously estimating the respective states via generalized feedback. The model at hand is motivated by a joint radar and communication system where radar and data applications share the same frequency band. For physically degraded BCs with i.i.d. state sequences, we characterize the capacity-distortion tradeoff region. For general BCs, we provide inner and outer bounds on the capacitydistortion region, as well as a sufficient condition when …
The asynchronous capacity region of memoryless multiple-access channels is the union of certain polytopes. It is well-known that vertices of such polytopes may be approached via a technique called successive decoding. It is also known that an extension of successive decoding applies to the dominant face of such polytopes. The extension consists of forming groups of users in such a way that users within a group are decoded jointly whereas groups are decoded successively. This paper goes one step further. It is shown that successive decoding extends to every face of the above mentioned polytopes…
This paper provides a numerical framework for computing the achievable rate region of memoryless multiple access channel (MAC) with a continuous alphabet from data. In particular, we use recent results on variational lower bounds on mutual information and KL-divergence to compute the boundaries of the rate region of MAC using a set of functions parameterized by neural networks. Our method relies on a variational lower bound on KL-divergence and an upper bound on KL-divergence based on the f-divergence inequalities. Unlike previous work, which computes an estimate on mutual information, which i…
The cognitive interference channel is an interference channel in which one transmitter is non-causally provided with the message of the other transmitter. This channel model has been extensively studied in the past years and capacity results for certain classes of channels have been proved. In this paper we present new inner and outer bounds for the capacity region of the cognitive interference channel as well as new capacity results. Previously proposed outer bounds are expressed in terms of auxiliary random variables for which no cardinality constraint is known. Consequently it is not possib…
We derive a new upper bound on the string oblivious transfer capacity of discrete memoryless channels. The main tool we use is the tension region of a pair of random variables introduced in Prabhakaran and Prabhakaran (2014) where it was used to derive upper bounds on rates of secure sampling in the source model. In this paper, we consider secure computation of string oblivious transfer in the channel model. Our bound is based on a monotonicity property of the tension region in the channel model. We show that our bound strictly improves upon the upper bound of Ahlswede and Csiszár (2013).
We study communication systems over state-dependent channels in the presence of a malicious state-aware jamming adversary. The channel has a memoryless state with an underlying distribution. The adversary introduces a jamming signal into the channel. The state sequence is known non-causally to both the encoder and the adversary. Taking an Arbitrarily Varying Channel (AVC) approach, we consider two setups, namely, the discrete memoryless Gel'fand-Pinsker (GP) AVC and the additive white Gaussian Dirty Paper (DP) AVC. We determine the randomized coding capacity of both the AVCs under a maximum pr…
Non-orthogonal communications play an important role in future communication architectures. In such scenarios, the received signal is corrupted by an interfering signal which is typically cyclostationary in continuous-time. If the period of interference is synchronized with the sampling period, the sampled interference is a discrete-time (DT) cyclostationary process. However, in the common interference scenario, the period of the interference is not necessarily synchronized with the sampling period. In such cases, the DT interference is an almost cyclostationary process. In this work we charac…
This paper investigates the first- and second-order maximum achievable rates of codes with/without cost constraints for mixed {channels} whose channel law is characterized by a general mixture of (at most) uncountably many stationary and memoryless discrete channels. These channels are referred to as {mixed memoryless channels with general mixture} and include the class of mixed memoryless channels of finitely or countably memoryless channels as a special case. For mixed memoryless channels with general mixture, the first-order coding theorem which gives a formula for the $\varepsilon$-capacit…
We show that the Extrinsic Information about the coded bits of any good (capacity achieving) code operating over a wide class of discrete memoryless channels (DMC) is zero when channel capacity is below the code rate and positive constant otherwise, that is, the Extrinsic Information Transfer (EXIT) chart is a step function of channel quality, for any capacity achieving code. It follows that, for a common class of iterative receivers where the error correcting decoder must operate at first iteration at rate above capacity (such as in turbo equalization, turbo channel estimation, parallel and s…
In this paper, we study the finite blocklength limits of state-dependent discrete memoryless channels where the discrete memoryless state is known noncausally at the encoder. For the point-to-point case, this is known as the Gel'fand-Pinsker channel model. We define the (n,ε)-capacity of the Gel'fand-Pinsker channel as the maximal rate of transmission of a message subject to the condition that the length of the block-code is n and the average error probability is no larger than ε. This paper provides a lower bound for the (n,ε)-capacity of the Gel'fand-Pinsker channel model, and hence an upper…
We study the problem of communicating over a discrete memoryless two-way channel using non-adaptive schemes, under a zero probability of error criterion. We derive single-letter inner and outer bounds for the zero-error capacity region, based on random coding, linear programming, linear codes, and the asymptotic spectrum of graphs. Among others, we provide a single-letter outer bound based on a combination of Shannon's vanishing-error capacity region and a two-way analogue of the linear programming bound for point-to-point channels, which in contrast to the one-way case, is generally better th…
We derive a lower and upper bound on the reliability function of discrete memoryless multiple-access channel (MAC) with noiseless feedback and variable-length codes (VLCs). For the upper-bound, we use proof techniques of Burnashev for the point-to-point case. Also, we adopt the techniques used to prove the converse for the feedback-capacity of MAC. For the lower-bound on the error exponent, we present a coding scheme consisting of a data and a confirmation stage. In the data stage, any arbitrary feedback capacity-achieving code is used. In the confirmation stage, each transmitter sends one bit…
Previous works established the capacity region for some special cases of discrete memoryless degraded cognitive interference channel (CIC) with unidirectional destination cooperation (UDC). In this letter, we characterize the capacity region of the general discrete memoryless degraded CIC-UDC. The obtained results imply that the capacity region is achieved by the Gel'fand-Pinsker coding at the cognitive transmitter, superposition coding at the primary transmitter and decode-and-forward at the relay. Furthermore, using this general result and a novel converse analysis, we establish the capacity…
An outer bound to the capacity region of the two-receiver discrete memoryless broadcast channel is given. The outer bound is tight for all cases where the capacity region is known. When specialized to the case of no common information, this outer bound is contained in the Korner-Marton outer bound. This containment is shown to be strict for the binary skew-symmetric broadcast channel. Thus, this outer bound is in general tighter than all other known outer bounds on the discrete memoryless broadcast channel.
A method is proposed, called channel polarization, to construct code sequences that achieve the symmetric capacity $I(W)$ of any given binary-input discrete memoryless channel (B-DMC) $W$. The symmetric capacity is the highest rate achievable subject to using the input letters of the channel with equal probability. Channel polarization refers to the fact that it is possible to synthesize, out of $N$ independent copies of a given B-DMC $W$, a second set of $N$ binary-input channels $\{W_N^{(i)}:1\le i\le N\}$ such that, as $N$ becomes large, the fraction of indices $i$ for which $I(W_N^{(i)})$ …
We address the problem of universal communications over an unknown channel with an instantaneous noiseless feedback, and show how rates corresponding to the empirical behavior of the channel can be attained, although no rate can be guaranteed in advance. First, we consider a discrete modulo-additive channel with alphabet $\mathcal{X}$, where the noise sequence $Z^n$ is arbitrary and unknown and may causally depend on the transmitted and received sequences and on the encoder's message, possibly in an adversarial fashion. Although the classical capacity of this channel is zero, we show that rate…
This paper derives an outer bound on the capacity region of a general memoryless interference channel with an arbitrary number of users. The derivation follows from a generalization of the techniques developed by Kramer and by Etkin et al for the Gaussian two-user channel. The derived bound is the first known outer bound valid for any memoryless channel. In Gaussian noise, classes of channels for which the proposed bound gives the sum-rate capacity are identified, including degraded channels and a class of Z-channels.
We consider the Gel'fand-Pinsker problem in which the channel and state are general, i.e., possibly non-stationary, non-memoryless and non-ergodic. Using the information spectrum method and a non-trivial modification of the piggyback coding lemma by Wyner, we prove that the capacity can be expressed as an optimization over the difference of a spectral inf- and a spectral sup-mutual information rate. We consider various specializations including the case where the channel and state are memoryless but not necessarily stationary.
Classically, communication systems are designed assuming perfect channel state information at the receiver and/or transmitter. However, in many practical situations, only an estimate of the channel is available that differs from the true channel. We address this channel mismatch scenario by using the notion of estimation-induced outage capacity, for which we provide an associated coding theorem and its strong converse, assuming a discrete memoryless channel. We illustrate our ideas via numerical simulations for transmissions over Ricean fading channels under a quality of service (QoS) constrai…
Channel polarization is a method of constructing capacity achieving codes for symmetric binary-input discrete memoryless channels (B-DMCs) [1]. In the original paper, the construction complexity is exponential in the blocklength. In this paper, a new construction method for arbitrary symmetric binary memoryless channel (B-MC) with linear complexity in the blocklength is proposed. Furthermore, new upper and lower bounds of the block error probability of polar codes are derived for the BEC and the arbitrary symmetric B-MC, respectively.
The traditional notion of capacity studied in the context of memoryless network communication builds on the concept of block-codes and requires that, for sufficiently large blocklength n, all receiver nodes simultaneously decode their required information after n channel uses. In this work, we generalize the traditional capacity region by exploring communication rates achievable when some receivers are required to decode their information before others, at different predetermined times; referred here as the "time-rate" region. Through a reduction to the standard notion of capacity, we present …
In this paper, we propose a novel joint coding-modulation technique based on serial concatenation of orthogonal linear transform, such as discrete Fourier transform (DFT) or Walsh-Hadamard transform (WHT), with memoryless nonlinearity. We demonstrate that such a simple signal construction may exhibit properties of a random code ensemble, as a result approaching channel capacity. Our computer simulations confirm that if the decoder relies on a modified approximate message passing algorithm, the proposed modulation technique exhibits performance on par with state-of-the-art coded modulation sche…
A method of channel polarization, proposed by Arikan, allows us to construct efficient capacity-achieving channel codes. In the original work, binary input discrete memoryless channels are considered. A special case of $q$-ary channel polarization is considered by Sasoglu, Telatar, and Arikan. In this paper, we consider more general channel polarization on $q$-ary channels. We further show explicit constructions using Reed-Solomon codes, on which asymptotically fast channel polarization is induced.
In this work we study the secrecy capacity of Gaussian multiple-input multiple-output (MIMO) wiretap channels (WTCs) with a finite memory, subject to a per-symbol average power constraint on the MIMO channel input. MIMO channels with finite memory are very common in wireless communications as well as in wireline communications (e.g., in communications over power lines). To derive the secrecy capacity of the Gaussian MIMO WTC with finite memory we first construct an asymptotically-equivalent block-memoryless MIMO WTC, which is then transformed into a set of parallel, independent, memoryless MIM…
Existing fixed-length feedback communication schemes are either specialized to particular channels (Schalkwijk--Kailath, Horstein), or apply to general channels but either have high coding complexity (block feedback schemes) or are difficult to analyze (posterior matching). This paper introduces a new fixed-length feedback coding scheme which achieves the capacity for all discrete memoryless channels, has an error exponent that approaches the sphere packing bound as the rate approaches the capacity, and has $O(n\log n)$ coding complexity. These benefits are achieved by judiciously combining fe…
A memoryless state-dependent multiple access channel (MAC) is considered where two transmitters wish to convey a respective message to a receiver while simultaneously estimating the respective channel state via generalized feedback. The scenario is motivated by a joint radar and communication system where the radar and data applications share the same bandwidth. An achievable capacity-distortion tradeoff region is derived that outperforms a resource-sharing scheme through a binary erasure MAC with binary states.
We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users' codebooks, i.e., they perform oblivious processing - The destination, or central processor, however, is informed about the users' codebooks. We establish a single-letter characterization of the capacity region of this model for a class of discrete memoryless channels in which the outputs at the relay…
