This paper presents a joint typicality framework for encoding and decoding nested linear codes for multi-user networks. This framework provides a new perspective on compute-forward within the context of discrete memoryless networks. In particular, it establishes an achievable rate region for computing the weighted sum of nested linear codewords over a discrete memoryless multiple-access channel (MAC). When specialized to the Gaussian MAC, this rate region recovers and improves upon the lattice-based compute-forward rate region of Nazer and Gastpar, thus providing a unified approach for discrete memoryless and Gaussian networks. Furthermore, this framework can be used to shed light on the joint decoding rate region for compute-forward, which is considered an open problem. Specifically, this work establishes an achievable rate region for simultaneously decoding two linear combinations of nested linear codewords from K senders.
Permutations of correlated sequences of random variables appear naturally in a variety of applications such as graph matching and asynchronous communications. In this paper, the asymptotic statistical behavior of such permuted sequences is studied. It is assumed that a collection of random vectors is produced based on an arbitrary joint distribution, and the vectors undergo a permutation operation. The joint typicality of the resulting permuted vectors with respect to the original distribution is investigated. As an initial step, permutations of pairs of correlated random vectors are considere…
In this paper, the authors report a way to use concepts from statistical learning to gain an advantage in terms of error exponents while communicating over a discrete memoryless channel. The study utilizes the simulation capability of the scientific computing package MATLAB to show that the proposed decoding method performs better than the traditional method of joint typicality decoding. The advantage is secured by modifying the traditional specification of what constitutes a decoding error. This is justified by the paradigm, also used in the program of `utilizing' noisy feedback, that one oug…
Let $\mathcal{X}$ and $\mathcal{Y}$ be finite alphabets and $P_{XY}$ a joint distribution over them, with $P_X$ and $P_Y$ representing the marginals. For any $ε> 0$, the set of $n$-length sequences $x^n$ and $y^n$ that are jointly typical \cite{ckbook} according to $P_{XY}$ can be represented on a bipartite graph. We present a formal definition of such a graph, known as a \emph{typicality} graph, and study some of its properties.
Consider a receiver in a multi-user network that wishes to decode several messages. Simultaneous joint typicality decoding is one of the most powerful techniques for determining the fundamental limits at which reliable decoding is possible. This technique has historically been used in conjunction with random i.i.d. codebooks to establish achievable rate regions for networks. Recently, it has been shown that, in certain scenarios, nested linear codebooks in conjunction with "single-user" or sequential decoding can yield better achievable rates. For instance, the compute-forward problem examines…
Consider the following information theoretic setup wherein independent codebooks of N correlated random variables are generated according to their respective marginals. The problem of determining the conditions on the rates of codebooks to ensure the existence of at least one codeword tuple which is jointly typical with respect to a given joint density (called the multivariate covering lemma) has been studied fairly well and the associated rate regions have found applications in several source coding scenarios. However, several multiterminal source coding applications, such as the general mult…
A fundamental tool to prove inner bounds in classical network information theory is the so-called conditional joint typicality lemma. In addition to the lemma, one often uses unions and intersections of typical sets in the inner bound arguments without so much as giving them a second thought. These arguments do not work in the quantum setting. This bottleneck shows up in the fact that so-called simultaneous decoders, as opposed to successive cancellation decoders, are known for very few channels in quantum network information theory. Another manifestation of this bottleneck is the lack of so-c…
The index coding problem studies the fundamental limit on broadcasting multiple messages to their respective receivers with different sets of side information that are represented by a directed graph. The generalized lexicographic product structure in the side information graph is introduced as a natural condition under which the corresponding index coding problem can be decomposed into multiple interacting subproblems. The capacity region is characterized in terms of the subproblem capacity regions combined in the same product structure. The proof is based on dual uses of random coding---one …
We prove new inner bounds for several multiterminal channels with classical inputs and quantum outputs. Our inner bounds are all proved in the one-shot setting, and are natural analogues of the best classical inner bounds for the respective channels. For some of these channels, similar quantum inner bounds were unknown even in the asymptotic iid setting. We prove our inner bounds by appealing to a new classical-quantum joint typicality lemma proved in a companion paper ("A one-shot quantum joint typicality lemma", Pranab Sen, arXiv:1806.07278). This lemma allows us to lift to the quantum setti…
This paper studies properties of binary runlength-limited sequences with additional constraints on their Hamming weight and/or their number of runs of identical symbols. An algebraic and a probabilistic (entropic) characterization of the exponential growth rate of the number of such sequences, i.e., their information capacity, are obtained by using the methods of multivariate analytic combinatorics, and properties of the capacity as a function of its parameters are stated. The second-order term in the asymptotic expansion of the rate of these sequences is also given, and the typical values of …
Compute-forward is a coding technique that enables receiver(s) in a network to directly decode one or more linear combinations of the transmitted codewords. Initial efforts focused on Gaussian channels and derived achievable rate regions via nested lattice codes and single-user (lattice) decoding as well as sequential (lattice) decoding. Recently, these results have been generalized to discrete memoryless channels via nested linear codes and joint typicality coding, culminating in a simultaneous-decoding rate region for recovering one or more linear combinations from $K$ users. Using a discret…
Two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel, in which a stochastic channel output is generated based on a compressed description of the channel input. Wyner's common information is the minimum description rate needed. However, when common randomness independent of the input is available, the necessary description rate reduces to Shannon's mutual information. This work characterizes the optimal trade-off between the amount of common randomness used and the required rate of description. We also in…
This paper proposes a novel technique to prove a one-shot version of achievability results in network information theory. The technique is not based on covering and packing lemmas. In this technique, we use an stochastic encoder and decoder with a particular structure for coding that resembles both the ML and the joint-typicality coders. Although stochastic encoders and decoders do not usually enhance the capacity region, their use simplifies the analysis. The Jensen inequality lies at the heart of error analysis, which enables us to deal with the expectation of many terms coming from stochast…
Nested codes have been employed in a large number of communication applications as a specific case of superposition codes, for example to implement binning schemes in the presence of noise, in joint network-channel coding, or in physical-layer secrecy. Whereas nested lattice codes have been proposed recently for continuous-input channels, in this paper we focus on the construction of nested linear codes for joint channel-network coding problems based on algebraic protograph LDPC codes. In particular, over the past few years several constructions of codes have been proposed that are based on ra…
We study a two-user state-dependent generalized multiple-access channel (GMAC) with correlated states. It is assumed that each encoder has \emph{noncausal} access to channel state information (CSI). We develop an achievable rate region by employing rate-splitting, block Markov encoding, Gelfand--Pinsker multicoding, superposition coding and joint typicality decoding. In the proposed scheme, the encoders use a partial decoding strategy to collaborate in the next block, and the receiver uses a backward decoding strategy with joint unique decoding at each stage. Our achievable rate region include…
This paper focuses on $ K $-receiver discrete-time memoryless broadcast channels (DM-BCs) with private messages, where the transmitter wishes to convey $K$ private messages to $K$ receivers respectively. A general inner bound on the capacity region is proposed based on an exhaustive message splitting and a $K$-level modified Marton's coding. The key idea is to split every message into $ \sum_{j=1}^K {K\choose j} $ submessages each corresponding to a set of users who are assigned to recover them, and then send these submessages through codewords that are jointly typical with each other. To guar…
Source localization and spectral estimation are among the most fundamental problems in statistical and array signal processing. Methods which rely on the orthogonality of the signal and noise subspaces, such as Pisarenko's method, MUSIC, and root-MUSIC are some of the most widely used algorithms to solve these problems. As a common feature, these methods require both apriori knowledge of the number of sources, and an estimate of the noise subspace. Both requirements are complicating factors to the practical implementation of the algorithms, and when not satisfied exactly, can potentially lead …
In this paper, we study the multiple-antenna wireless communication networks, where a large number of devices simultaneously communicate with an access point. The capacity region of multiple-input multiple-output massive multiple access channels (MIMO mMAC) is investigated. While joint typicality decoding is utilized to establish the achievability of capacity region for conventional MAC with fixed number of users, the technique is not directly applicable for MIMO mMAC. Instead, an information-theoretic approach based on Gallager's error exponent analysis is exploited to characterize the \textc…
Practical constructions of lossless distributed source codes (for the Slepian-Wolf problem) have been the subject of much investigation in the past decade. In particular, near-capacity achieving code designs based on LDPC codes have been presented for the case of two binary sources, with a binary-symmetric correlation. However, constructing practical codes for the case of non-binary sources with arbitrary correlation remains by and large open. From a practical perspective it is also interesting to consider coding schemes whose performance remains robust to uncertainties in the joint distributi…
In this paper, we take a unified approach for network information theory and prove a coding theorem, which can recover most of the achievability results in network information theory that are based on random coding. The final single-letter expression has a very simple form, which was made possible by many novel elements such as a unified framework that represents various network problems in a simple and unified way, a unified coding strategy that consists of a few basic ingredients but can emulate many known coding techniques if needed, and new proof techniques beyond the use of standard cover…
In this paper, we want to investigate classical-quantum multiple access wiretap channels (CQ-MA-WTC) under one-shot setting. In this regard, we analyze the CQ-MA-WTC using simultaneous position-based decoder for reliable decoding and using a newly introduced technique in order to decode securely. Also, for the sake of comparison, we analyze the CQ-MA-WTC using Sen's one-shot joint typicality lemma for reliable decoding. The simultaneous position-based decoder tends to a multiple hypothesis testing problem. Also, using convex splitting to analyze the privacy criteria in a simultaneous scenario …
Since the publication of Shannon's theory of one terminal source coding, a number of interesting extensions have been derived by researchers such as Slepian-Wolf, Wyner, Ahlswede-Körner, Wyner-Ziv and Berger-Yeung. Specifically, the achievable rate or rate-distortion region has been described by a first order information-theoretic functional of the source statistics in each of the above cases. At the same time several problems have also remained unsolved. Notable two terminal examples include the joint distortion problem, where both sources are reconstructed under a combined distortion criteri…
A noisy network coding scheme for sending multiple sources over a general noisy network is presented. For multi-source multicast networks, the scheme naturally extends both network coding over noiseless networks by Ahlswede, Cai, Li, and Yeung, and compress-forward coding for the relay channel by Cover and El Gamal to general discrete memoryless and Gaussian networks. The scheme also recovers as special cases the results on coding for wireless relay networks and deterministic networks by Avestimehr, Diggavi, and Tse, and coding for wireless erasure networks by Dana, Gowaikar, Palanki, Hassibi,…
