The exact order of the optimal sub-exponentially decaying factor in the classical bounds on the error probability of fixed-length codes over a Gallager-symmetric discrete memoryless channel with and without ideal feedback is determined. Regardless of the availability of feedback, it is shown that the order of the optimal sub-exponential factor exhibits a dichotomy. Moreover, the proof technique is used to establish the third-order term in the normal approximation for symmetric channels, where a similar dichotomy is shown to exist.
Multilevel diversity coding is a classical coding model where multiple mutually independent information messages are encoded, such that different reliability requirements can be afforded to different messages. It is well known that {\em superposition coding}, namely separately encoding the independent messages, is optimal for symmetric multilevel diversity coding (SMDC) (Yeung-Zhang 1999). In the current paper, we consider weakly secure SMDC where security constraints are injected on each individual message, and provide a complete characterization of the conditions under which superposition co…
It is well known that {\em superposition coding}, namely separately encoding the independent sources, is optimal for symmetric multilevel diversity coding (SMDC) (Yeung-Zhang 1999). However, the characterization of the coding rate region therein involves uncountably many linear inequalities and the constant term (i.e., the lower bound) in each inequality is given in terms of the solution of a linear optimization problem. Thus this implicit characterization of the coding rate region does not enable the determination of the achievability of a given rate tuple. In this paper, we first obtain clos…
Universal fixed-to-variable lossless source coding for memoryless sources is studied in the finite blocklength and higher-order asymptotics regimes. Optimal third-order coding rates are derived for general fixed-to-variable codes and for prefix codes. It is shown that the non-prefix Type Size code, in which codeword lengths are chosen in ascending order of type class size, achieves the optimal third-order rate and outperforms classical Two-Stage codes. Converse results are proved making use of a result on the distribution of the empirical entropy and Laplace's approximation. Finally, the fixed…
In this paper, we propose a new coding scheme for symmetric Gaussian interference channels with feedback based on the ideas of time-varying coding schemes. The proposed scheme improves the Suh-Tse and Kramer inner bounds of the channel capacity for the cases of weak and not very strong interference. This improvement is more significant when the signal-to-noise ratio (SNR) is not very high. It is shown theoretically and numerically that our coding scheme can outperform the Kramer code. In addition, the generalized degrees-of-freedom of our proposed coding scheme is equal to the Suh-Tse scheme i…
This work presents a polar coding scheme for fading channels, focusing primarily on fading binary symmetric and additive exponential noise channels. For fading binary symmetric channels, a hierarchical coding scheme is presented, utilizing polar coding both over channel uses and over fading blocks. The receiver uses its channel state information (CSI) to distinguish states, thus constructing an overlay erasure channel over the underlying fading channels. By using this scheme, the capacity of a fading binary symmetric channel is achieved without CSI at the transmitter. Noting that a fading AWGN…
This paper considers error probabilities of random codes for memoryless channels in the fixed-rate regime. Random coding is a fundamental scheme to achieve the channel capacity and many studies have been conducted for the asymptotics of the decoding error probability. Gallager derived the exact asymptotics (that is, a bound with asymptotically vanishing relative error) of the error probability below the critical rate. On the other hand, exact asymptotics for fixed rate above the critical rate has been unknown except for symmetric channels (in the strong sense) and strongly nonlattice channels.…
An index coding problem with $n$ messages has symmetric rate $R$ if all $n$ messages can be conveyed at rate $R$. In a recent work, a class of index coding problems for which symmetric rate $\frac{1}{3}$ is achievable was characterised using special properties of the side-information available at the receivers. In this paper, we show a larger class of index coding problems (which includes the previous class of problems) for which symmetric rate $\frac{1}{3}$ is achievable. In the process, we also obtain a stricter necessary condition for rate $\frac{1}{3}$ feasibility than what is known in lit…
We present novel non-asymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include (i) the Wyner-Ahlswede-Korner (WAK) problem of almost-lossless source coding with rate-limited side-information, (ii) the Wyner-Ziv (WZ) problem of lossy source coding with side-information at the decoder and (iii) the Gel'fand-Pinsker (GP) problem of channel coding with noncausal state information available at the encoder. The bounds are proved using ideas from channel simulation and channel resolvability. Our bounds for all three problems…
A general time-varying feedback coding scheme is proposed for $M$-user fully connected symmetric Gaussian interference channels. Based on the analysis of the general coding scheme, we prove a theorem which gives a criterion for designing good time-varying feedback codes for Gaussian interference channels. The proposed scheme improves the Suh-Tse and Kramer inner bounds of the channel capacity for the cases of weak and not very strong interference when $M=2$. This capacity improvement is more significant when the signal-to-noise ratio (SNR) is not very high. In addition, our coding scheme can b…
In this paper, we revisit two multi-terminal lossy source coding problems: the lossy source coding problem with side information available at the encoder and one of the two decoders, which we term as the Kaspi problem (Kaspi, 1994), and the multiple description coding problem with one semi-deterministic distortion measure, which we refer to as the Fu-Yeung problem (Fu and Yeung, 2002). For the Kaspi problem, we first present the properties of optimal test channels. Subsequently, we generalize the notion of the distortion-tilted information density for the lossy source coding problem to the Kas…
A single unicast index coding problem (SUICP) with symmetric neighboring interference (SNI) has equal number of $K$ messages and $K$ receivers, the $k$th receiver $R_{k}$ wanting the $k$th message $x_{k}$ and having the side-information $\mathcal{K}_{k}=(\mathcal{I}_{k} \cup x_{k})^c,$ where ${I}_k= \{x_{k-U},\dots,x_{k-2},x_{k-1}\}\cup\{x_{k+1}, x_{k+2},\dots,x_{k+D}\}$ is the interference with $D$ messages after and $U$ messages before its desired message. Maleki, Cadambe and Jafar obtained the capacity of this symmetric neighboring interference single unicast index coding problem (SNI-SUICP…
Tight bounds for several symmetric divergence measures are derived in terms of the total variation distance. It is shown that each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these bounds for lossless source coding is provided, refining and improving a certain bound by Csiszár. Another application of these bounds has been recently introduced by Yardi. et al. for channel-code detection.
In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth Renyi entropy characterizes the optimal exponential moment of the codeword length.
Error probabilities of random codes for memoryless channels are considered in this paper. In the area of communication systems, admissible error probability is very small and it is sometimes more important to discuss the relative gap between the achievable error probability and its bound than to discuss the absolute gap. Scarlett et al. derived a good upper bound of a random coding union bound based on the technique of saddlepoint approximation but it is not proved that the relative gap of their bound converges to zero. This paper derives a new bound on the achievable error probability in this…
Consider a generalized multiterminal source coding system, where $\ell\choose m$ encoders, each observing a distinct size-$m$ subset of $\ell$ ($\ell\geq 2$) zero-mean unit-variance symmetrically correlated Gaussian sources with correlation coefficient $ρ$, compress their observations in such a way that a joint decoder can reconstruct the sources within a prescribed mean squared error distortion based on the compressed data. The optimal rate-distortion performance of this system was previously known only for the two extreme cases $m=\ell$ (the centralized case) and $m=1$ (the distributed case)…
In this paper we provide new compact integral expressions and associated simple asymptotic approximations for converse and achievability bounds in the finite blocklength regime. The chosen converse and random coding union bounds were taken from the recent work of Polyanskyi-Poor-Verdu, and are investigated under parallel AWGN channels, the AWGN channels, the BI-AWGN channel, and the BSC. The technique we use, which is a generalization of some recent results available from the literature, is to map the probabilities of interest into a Laplace integral, and then solve (or approximate) the integr…
A single unicast index coding problem (SUICP) with symmetric and consecutive interference (SCI) has $K$ messages and $K$ receivers, the $k$th receiver $R_k$ wanting the $k$th message $x_k$ and having interference $\mathcal{I}_k= \{x_{k-U-m},\dots,x_{k-m-2},x_{k-m-1}\}\cup\{x_{k+m+1}, x_{k+m+2},\dots,x_{k+m+D}\}$ and side-information $\mathcal{K}_k=(\mathcal{I}_k \cup x_k)^c$. In this paper, we derive a lowerbound on the broadcast rate of single unicast index coding problem with symmetric and consecutive interference (SUICP(SCI)). In the SUICP(SCI), if $m=0$, we refer this as single unicast ind…
The zero-error channel capacity is the maximum asymptotic rate that can be reached with error probability exactly zero, instead of a vanishing error probability. The nature of this problem, essentially combinatorial rather than probabilistic, has led to various researches both in Information Theory and Combinatorics. However, the zero-error capacity is still an open problem, for example the capacity of the noisy-typewriter channel with 7 letters is unknown. In this article, we propose a new approach to construct optimal zero-error codes, based on the concatenation of words of variable-length, …
This paper characterizes the second-order coding rates for lossy source coding with side information available at both the encoder and the decoder. We first provide non-asymptotic bounds for this problem and then specialize the non-asymptotic bounds for three different scenarios: discrete memoryless sources, Gaussian sources, and Markov sources. We obtain the second-order coding rates for these settings. It is interesting to observe that the second-order coding rate for Gaussian source coding with Gaussian side information available at both the encoder and the decoder is the same as that for G…
A single unicast index coding problem (SUICP) with symmetric neighboring and consecutive side-information (SNCS) has $K$ messages and $K$ receivers, the $k$th receiver $R_k$ wanting the $k$th message $x_k$ and having the side-information $\mathcal{K}_k=\{x_{k-U},\dots,x_{k-2},x_{k-1}\}\cup\{x_{k+1}, x_{k+2},\dots,x_{k+D}\}$. The single unicast index coding problem with symmetric neighboring and consecutive side-information, SUICP(SNCS), is motivated by topological interference management problems in wireless communication networks. Maleki, Cadambe and Jafar obtained the symmetric capacity of t…
Compute-and-forward relaying is effective to increase bandwidth efficiency of wireless two-way relay channels. In a compute-and-forward scheme, a relay tries to decode a linear combination composed of transmitted messages from other terminals or relays. Design for error correcting codes and its decoding algorithms suitable for compute-and-forward relaying schemes are still important issue to be studied. In this paper, we will present an asymptotic performance analysis on LDPC codes over two-way relay channels based on density evolution (DE). Because of the asymmetric nature of the channel, we …
The capacity of symmetric instance of the multiple unicast index coding problem with neighboring antidotes (side-information) with number of messages equal to the number of receivers was given by Maleki, Cadambe and Jafar. In this paper we consider ten symmetric multiple unicast problems with lesser antidotes than considered by them and explicitly construct scalar linear codes for these problems. These codes are shown to achieve the capacity or equivalently these codes shown to be of optimal length. Also, the constructed codes enable the receivers use small number of transmissions to decode th…
Index coding models broadcast networks in which a sender sends different messages to different receivers simultaneously, where each receiver may know some of the messages a priori. The aim is to find the minimum (normalised) index codelength that the sender sends. This paper considers unicast index coding, where each receiver requests exactly one message, and each message is requested by exactly one receiver. Each unicast index-coding instances can be fully described by a directed graph and vice versa, where each vertex corresponds to one receiver. For any directed graph representing a unicast…
A single unicast index coding problem (SUICP) with symmetric neighboring interference (SNI) has equal number of $K$ messages and $K$ receivers, the $k$th receiver $R_{k}$ wanting the $k$th message $x_{k}$ and having the side-information $\mathcal{K}_{k}=(\mathcal{I}_{k} \cup x_{k})^c,$ where ${I}_k= \{x_{k-U},\dots,x_{k-2},x_{k-1}\}\cup\{x_{k+1}, x_{k+2},\dots,x_{k+D}\}$ is the interference with $D$ messages after and $U$ messages before its desired message. The single unicast index coding problem with symmetric neighboring interference (SUICP-SNI) is motivated by topological interference mana…
Koetter和Kschischang在[R. Koetter和F.R. Kschischang，“Coding for Errors and Erasures in Random Network Coding”，IEEE Trans. Inform.理论，{54（8），2008]，Gabidulin码的网络编码对应物相对于子空间距离执行渐近最优。最近，Silva和Kschischang在[D.席尔瓦和F.R. Kschischang，“On Eschem for Error Correction in Network Coding”，将出现在IEEE Trans. Inform. Theory，ArXiv：0805.3824v4[cs.IT]，2009]，注入距离给出了在非相干网络编码中发生的事情的详细图片。我们表明，上述代码也是渐近最优的关于这个距离。
