Ahlswede and Dueck showed possibility to identify with high probability one out of $M$ messages by transmitting $1/C\log\log M$ bits only, where $C$ is the channel capacity. It is known that this identification can be based on error-correcting codes. We propose an identification procedure based on random codes that achieves channel capacity. Then we show that this procedure can be simplified using pseudo-random generators.
In the case of ordinary identification coding, a code is devised to identify a single object among $N$ objects. But, in this paper, we consider an identification coding problem to identify $K$ objects at once among $N$ objects in the both cases that $K$ objects are ranked or not ranked. By combining Kurosawa-Yoshida scheme with Moulin-Koetter scheme, an efficient identification coding scheme is proposed, which can attain high coding rate and error exponents compared with the case that an ordinary identification code is used $K$ times. Furthermore, the achievable triplet of rate and error expon…
Beckman Institute（贝克曼研究所）；University of Illinois（伊利诺伊大学）
The problem of joint universal source coding and modeling, treated in the context of lossless codes by Rissanen, was recently generalized to fixed-rate lossy coding of finitely parametrized continuous-alphabet i.i.d. sources. We extend these results to variable-rate lossy block coding of stationary ergodic sources and show that, for bounded metric distortion measures, any finitely parametrized family of stationary sources satisfying suitable mixing, smoothness and Vapnik-Chervonenkis learnability conditions admits universal schemes for joint lossy source coding and identification. We also give…
We derive various error exponents in the bee identification problem under two different decoding rules. Under naïve decoding, which decodes each bee independently of the others, we analyze a general discrete memoryless channel and a relatively wide family of stochastic decoders. Upper and lower bounds to the random coding error exponent are derived and proved to be equal at relatively high coding rates. Then, we propose a lower bound on the error exponent of the typical random code, which improves upon the random coding exponent at low coding rates. We also derive a third bound, which is relat…
The problem of joint universal source coding and modeling, addressed by Rissanen in the context of lossless codes, is generalized to fixed-rate lossy coding of continuous-alphabet memoryless sources. We show that, for bounded distortion measures, any compactly parametrized family of i.i.d. real vector sources with absolutely continuous marginals (satisfying appropriate smoothness and Vapnik--Chervonenkis learnability conditions) admits a joint scheme for universal lossy block coding and parameter estimation, and give nonasymptotic estimates of convergence rates for distortion redundancies and …
A randomized covering-packing duality between source and channel coding will be discussed by considering the source coding problem of coding a source with a certain distortion level and by considering a channel which communicates the source within a certain distortion level. An operational view of source-channel separation for communication with a fidelity criterion will be discussed in brief.
Proving capacity for networks without feedback or cooperation usually involves two fundamental random coding techniques: superposition coding and binning. Although conceptually very different, these two techniques often achieve the same performance, suggesting an underlying similarity. In this correspondence we propose a new random coding technique that generalizes superposition coding and binning and provides new insight on relationship among the two With this new theoretical tool, we derive new achievable regions for three classical information theoretical models: multi-access channel, broad…
Here we write in a unified fashion (using "R(P, Q, D)") the random coding exponents in channel coding and lossy source coding. We derive their explicit forms and show, that, for a given random codebook distribution Q, the channel decoding error exponent can be viewed as an encoding success exponent in lossy source coding, and the channel correct-decoding exponent can be viewed as an encoding failure exponent in lossy source coding. We then extend the channel exponents to arbitrary D, which corresponds for D > 0 to erasure decoding and for D < 0 to list decoding. For comparison, we also derive …
Random linear network coding is a feasible encoding tool for network coding, specially for the non-coherent network, and its performance is important in theory and application. In this letter, we study the performance of random linear network coding for the well-known butterfly network by analyzing the failure probabilities. We determine the failure probabilities of random linear network coding for the well-known butterfly network and the butterfly network with channel failure probability p.
In 1973, Gallager proved that the random-coding bound is exponentially tight for the random code ensemble at all rates, even below expurgation. This result explained that the random-coding exponent does not achieve the expurgation exponent due to the properties of the random ensemble, irrespective of the utilized bounding technique. It has been conjectured that this same behavior holds true for a random ensemble of linear codes. This conjecture is proved in this paper. Additionally, it is shown that this property extends to Poltyrev's random-coding exponent for a random ensemble of lattices.
In this work we present error-correcting codes for random network coding based on rank- metric codes, Ferrers diagrams, and puncturing. For most parameters, the constructed codes are larger than all previously known codes.
Beckman Institute for Advanced Science and Technology, University of Illinois（伊利诺伊大学贝克曼先进科学技术研究所）；Duke University（杜克大学）
We consider the problem of joint universal variable-rate lossy coding and identification for parametric classes of stationary $β$-mixing sources with general (Polish) alphabets. Compression performance is measured in terms of Lagrangians, while identification performance is measured by the variational distance between the true source and the estimated source. Provided that the sources are mixing at a sufficiently fast rate and satisfy certain smoothness and Vapnik-Chervonenkis learnability conditions, it is shown that, for bounded metric distortions, there exist universal schemes for joint los…
In this work, we derive the random coding error exponent for the uplink phase of a two-way relay system where physical layer network coding (PNC) is employed. The error exponent is derived for the practical (yet sub-optimum) XOR channel decoding setting. We show that the random coding error exponent under optimum (i.e., maximum likelihood) PNC channel decoding can be achieved even under the sub-optimal XOR channel decoding. The derived achievability bounds provide us with valuable insight and can be used as a benchmark for the performance of practical channel-coded PNC systems employing low co…
This purpose of this letter is to handle a gap that was found in the proof of Theorem 2 in the paper "The generalized stochastic likelihood decoder: random coding and expurgated bounds."
We complete the proof of conjecture, which allows to complete the derivation of the random coding bound for the reliability function in quantum channel in the case of arbitrary signal states
This work studies point-to-point, multiple access, and random access lossless source coding in the finite-blocklength regime. In each scenario, a random coding technique is developed and used to analyze third-order coding performance. Asymptotic results include a third-order characterization of the Slepian-Wolf rate region with an improved converse that relies on a connection to composite hypothesis testing. For dependent sources, the result implies that the independent encoders used by Slepian-Wolf codes can achieve the same third-order-optimal performance as a single joint encoder. The conce…
The problem of joint universal source coding and identification is considered in the setting of fixed-rate lossy coding of continuous-alphabet memoryless sources. For a wide class of bounded distortion measures, it is shown that any compactly parametrized family of $\R^d$-valued i.i.d. sources with absolutely continuous distributions satisfying appropriate smoothness and Vapnik--Chervonenkis learnability conditions, admits a joint scheme for universal lossy block coding and parameter estimation, such that when the block length $n$ tends to infinity, the overhead per-letter rate and the distort…
This paper studies the subexponential prefactor to the random-coding bound for a given rate. Using a refinement of Gallager's bounding techniques, an alternative proof of a recent result by Altuğ and Wagner is given, and the result is extended to the setting of mismatched decoding.
We consider ensembles of channel codes that are partitioned into bins, and focus on analysis of exact random coding error exponents associated with optimum decoding of the index of the bin to which the transmitted codeword belongs. Two main conclusions arise from this analysis: (i) for independent random selection of codewords within a given type class, the random coding exponent of optimal bin index decoding is given by the ordinary random coding exponent function, computed at the rate of the entire code, independently of the exponential rate of the size of the bin. (ii) for this ensemble of …
Consider a lossy communication channel for unicast with zero-delay feedback. For this communication scenario, a simple retransmission scheme is optimum with respect to delay. An alternative approach is to use random linear coding in automatic repeat-request (ARQ) mode. We extend the work of Shrader and Ephremides, by deriving an expression for the delay of random linear coding over field of infinite size. Simulation results for various field sizes are also provided.
This paper studies multiuser random coding techniques for channel coding with a given (possibly suboptimal) decoding rule. For the mismatched discrete memoryless multiple-access channel, an error exponent is obtained that is tight with respect to the ensemble average, and positive within the interior of Lapidoth's achievable rate region. This exponent proves the ensemble tightness of the exponent of Liu and Hughes in the case of maximum-likelihood decoding. An equivalent dual form of Lapidoth's achievable rate region is given, and the latter is shown to extend immediately to channels with infi…
An improved pre-factor for the random coding bound is proved. Specifically, for channels with critical rate not equal to capacity, if a regularity condition is satisfied (resp. not satisfied), then for any $ε>0$ a pre-factor of $O(N^{-\frac{1}{2}\left( 1 - ε+ \barρ^\ast_R \right)})$ (resp. $O(N^{-\frac{1}{2}})$) is achievable for rates above the critical rate, where $N$ and $R$ is the blocklength and rate, respectively. The extra term $\barρ^\ast_R$ is related to the slope of the random coding exponent. Further, the relation of these bounds with the authors' recent refinement of the sphere-pac…
Our primary goal in this paper is to traverse the performance gap between two linear network coding schemes: random linear network coding (RLNC) and instantly decodable network coding (IDNC) in terms of throughput and decoding delay. We first redefine the concept of packet generation and use it to partition a block of partially-received data packets in a novel way, based on the coding sets in an IDNC solution. By varying the generation size, we obtain a general coding framework which consists of a series of coding schemes, with RLNC and IDNC identified as two extreme cases. We then prove that …
Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels a…
This paper presents a saddlepoint approximation of the random-coding union bound of Polyanskiy et al. for i.i.d. random coding over discrete memoryless channels. The approximation is single-letter, and can thus be computed efficiently. Moreover, it is shown to be asymptotically tight for both fixed and varying rates, unifying existing achievability results in the regimes of error exponents, second-order coding rates, and moderate deviations. For fixed rates, novel exact-asymptotics expressions are specified to within a multiplicative 1+o(1) term. A numerical example is provided for which the a…
A new universal coding/decoding scheme for random access with collision detection is given in the case of two senders. The result is used to give an achievable joint source-channel coding error exponent for multiple access channels in the case of independent sources. This exponent is improved in a modified model that admits error free 0 rate communication between the senders.
Stochastic encoders for channel coding and lossy source coding are introduced with a rate close to the fundamental limits, where the only restriction is that the channel input alphabet and the reproduction alphabet of the lossy source code are finite. Random numbers, which satisfy a condition specified by a function and its value, are used to construct stochastic encoders. The proof of the theorems is based on the hash property of an ensemble of functions, where the results are extended to general channels/sources and alternative formulas are introduced for channel capacity and the rate-distor…
In this paper, we study network coding capacity for random wireless networks. Previous work on network coding capacity for wired and wireless networks have focused on the case where the capacities of links in the network are independent. In this paper, we consider a more realistic model, where wireless networks are modeled by random geometric graphs with interference and noise. In this model, the capacities of links are not independent. We consider two scenarios, single source multiple destinations and multiple sources multiple destinations. In the first scenario, employing coupling and martin…
We give a complete characterization of the correlated random coding secrecy capacity of arbitrarily varying wiretap channels (AVWCs). We apply two alternative strong secrecy criteria, which both lead to the same multi-letter formula. The difference of these criteria lies in the treatment of correlated randomness, they coincide in the case of uncorrelated codes. On the basis of the derived formula, we show that the correlated random coding secrecy capacity is continuous as a function of the AVWC, in contrast to the discontinuous uncorrelated coding secrecy capacity. In the proof of the secrecy …
This work contains two main contributions concerning the asymmetric broadcast channel. The first is an analysis of the exact random coding error exponents for both users, and the second is the derivation of universal decoders for both users. These universal decoders are certain variants of the maximum mutual information (MMI) universal decoder, which achieve the corresponding random coding exponents of optimal decoding. In addition, we introduce some lower bounds, which involve optimization over very few parameters, unlike the original, exact exponents, which involve minimizations over auxilia…
In this paper we consider the identification (ID) via multiple access channels (MACs). In the general MAC the ID capacity region includes the ordinary transmission (TR) capacity region. In this paper we discuss the converse coding theorem. We estimate two types of error probabilities of identification for rates outside capacity region, deriving some function which serves as a lower bound of the sum of two error probabilities of identification. This function has a property that it tends to zero as $n\to \infty$ for noisy channels satisfying the strong converse property. Using this property, we …
This paper is about deriving lower bounds on the error exponents for the two-user interference channel under the random coding regime for several ensembles. Specifically, we first analyze the standard random coding ensemble, where the codebooks are comprised of independently and identically distributed (i.i.d.) codewords. For this ensemble, we focus on optimum decoding, which is in contrast to other, suboptimal decoding rules that have been used in the literature (e.g., joint typicality decoding, treating interference as noise, etc.). The fact that the interfering signal is a codeword, rather …
With the phenomenal growth of the Internet of Things (IoT), Ultra Reliable Low Latency Communications (URLLC) has potentially been the enabler to guarantee the stringent requirements on latency and reliability. However, how to achieve low latency and ultra-reliability with the random arrival remains open. In this paper, a queue-aware variable-length channel coding is presented over the single URLLC user link, in which the finite blocklength of channel coding is determined based on the random arrival. More particularly, a cross-layer approach is proposed for the URLLC user to establish the opti…
We study the performance of random linear network coding for time division duplexing channels with Poisson arrivals. We model the system as a bulk-service queue with variable bulk size. A full characterization for random linear network coding is provided for time division duplexing channels [1] by means of the moment generating function. We present numerical results for the mean number of packets in the queue and consider the effect of the range of allowable bulk sizes. We show that there exists an optimal choice of this range that minimizes the mean number of data packets in the queue.
In this paper we explore tradeoffs, regarding coding performance, between the thickness and spacing of the cutset used in Reduced Cutset Coding (RCC) of a Markov random field image model. Considering MRF models on a square lattice of sites, we show that under a stationarity condition, increasing the thickness of the cutset reduces coding rate for the cutset, increasing the spacing between components of the cutset increases the coding rate of the non-cutset pixels, though the coding rate of the latter is always strictly less than that of the former. We show that the redundancy of RCC can be dec…
Error probabilities of random codes for memoryless channels are considered in this paper. In the area of communication systems, admissible error probability is very small and it is sometimes more important to discuss the relative gap between the achievable error probability and its bound than to discuss the absolute gap. Scarlett et al. derived a good upper bound of a random coding union bound based on the technique of saddlepoint approximation but it is not proved that the relative gap of their bound converges to zero. This paper derives a new bound on the achievable error probability in this…
Random linear network coding is a particularly decentralized approach to the multicast problem. Use of random network codes introduces a non-zero probability however that some sinks will not be able to successfully decode the required sources. One of the main theoretical motivations for random network codes stems from the lower bound on the probability of successful decoding reported by Ho et. al. (2003). This result demonstrates that all sinks in a linearly solvable network can successfully decode all sources provided that the random code field size is large enough. This paper develops a new …
We establish a random variable substitution lemma and use it to investigate the role of refinement layer in multiple description coding, which clarifies the relationship among several existing achievable multiple description rate-distortion regions. Specifically, it is shown that the El Gamal-Cover (EGC) region is equivalent to the EGC* region (an antecedent version of the EGC region) while the Venkataramani-Kramer-Goyal (VKG) region (when specialized to the 2-description case) is equivalent to the Zhang-Berger (ZB) region. Moreover, we prove that for multiple description coding with individua…
In this paper, we consider noncoherent random linear coding networks (RLCNs) as a discrete memoryless channel (DMC) whose input and output alphabets consist of subspaces. This contrasts with previous channel models in the literature which assume matrices as the channel input and output. No particular assumptions are made on the network topology or the transfer matrix, except that the latter may be rank-deficient according to some rank deficiency probability distribution. We introduce a random vector basis selection procedure which renders the DMC symmetric. The capacity we derive can be seen a…
It is already known that in multicast (single source, multiple sinks) network, random linear network coding can achieve the maximum flow upper bound. In this paper, we investigate how random linear network coding behaves in general multi-source multi-sink case, where each sink has different demands, and characterize all achievable rate of random linear network coding by a simple maximum flow condition.
Leveraging recent progress in physical-layer network coding we propose a new approach to random access: When packets collide, it is possible to recover a linear combination of the packets at the receiver. Over many rounds of transmission, the receiver can thus obtain many linear combinations and eventually recover all original packets. This is by contrast to slotted ALOHA where packet collisions lead to complete erasures. The throughput of the proposed strategy is derived and shown to be significantly superior to the best known strategies, including multipacket reception.
New non-asymptotic random coding theorems (with error probability $ε$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input arbitrary output channels. The resulting non-asymptotic achievability bounds, when combined with non-asymptotic equipartition properties developed in the paper, can be easily computed. Analytically, these non-asymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as $n$ goes to infinity with either constant o…
The likelihood decoder is a stochastic decoder that selects the decoded message at random, using the posterior distribution of the true underlying message given the channel output. In this work, we study a generalized version of this decoder where the posterior is proportional to a general function that depends only on the joint empirical distribution of the output vector and the codeword. This framework allows both mismatched versions and universal (MMI) versions of the likelihood decoder, as well as the corresponding ordinary deterministic decoders, among many others. We provide a direct ana…
This paper considers the achievability and converse bounds on the maximal channel coding rate at a given blocklength and error probability over AWGN channels. The problem stems from covert communication with Gaussian codewords. By re-visiting [18], we first present new and more general achievability bounds for random coding schemes under maximal or average probability of error requirements. Such general bounds are then applied to covert communication in AWGN channels where codewords are generated from Gaussian distribution while meeting the maximal power constraint. Further comparison is made …
Flash memories intended for SSD and mobile applications need to provide high random I/O performance. This requires using efficient schemes for reading small chunks of data (e.g. 0.5KB - 4KB) from random addresses. Furthermore, in order to be cost efficient, it is desirable to use high density Multi-Level Cell (MLC) memories, such as the ones based on 3 or 4 bit per cell technologies. Unfortunately, these two requirements are contradicting, as reading an MLC memory, whose data is coded conventionally, requires multiple sensing operations, resulting in slow reading and degraded random I/O perfor…
We develop a low-complexity polar coding scheme for the discrete memoryless broadcast channel with confidential messages under strong secrecy and randomness constraints. Our scheme extends previous work by using an optimal rate of uniform randomness in the stochastic encoder, and avoiding assumptions regarding the symmetry or degraded nature of the channels. The price paid for these extensions is that the encoder and decoders are required to share a secret seed of negligible size and to increase the block length through chaining. We also highlight a close conceptual connection between the prop…
In practice, since many communication networks are huge in scale or complicated in structure even dynamic, the predesigned network codes based on the network topology is impossible even if the topological structure is known. Therefore, random linear network coding was proposed as an acceptable coding technique. In this paper, we further study the performance of random linear network coding by analyzing the failure probabilities at sink node for different knowledge of network topology and get some tight and asymptotically tight upper bounds of the failure probabilities. In particular, the worst…
We assess the practicality of random network coding by illuminating the issue of overhead and considering it in conjunction with increasingly long packets sent over the erasure channel. We show that the transmission of increasingly long packets, consisting of either of an increasing number of symbols per packet or an increasing symbol alphabet size, results in a data rate approaching zero over the erasure channel. This result is due to an erasure probability that increases with packet length. Numerical results for a particular modulation scheme demonstrate a data rate of approximately zero for…
For communications in the presence of eavesdroppers, random components are often used in code design to camouflage information from eavesdroppers. In broadcast channels without eavesdroppers, Marton error-correcting coding comprises random components which allow correlation between auxiliary random variables representing independent messages. In this paper, we study if Marton coding alone can ensure individual secrecy in the two-receiver discrete memoryless broadcast channel with a passive eavesdropper. Our results show that in accordance to the principle of Wyner secrecy coding, this is possi…
Randomized network coding (RNC) greatly reduces the complexity of implementing network coding in large-scale, heterogeneous networks. This paper examines two tradeoffs in applying RNC: The first studies how the performance of RNC varies with a node's randomizing capabilities. Specifically, a limited randomized network coding (L-RNC) scheme - in which intermediate nodes perform randomized encoding based on only a limited number of random coefficients - is proposed and its performance bounds are analyzed. Such a L-RNC approach is applicable to networks in which nodes have either limited computat…
It is common in the study of secure multicast network coding in the presence of an eavesdropper that has access to $z$ network links, to assume that the source node is the only node that generates random keys. In this setting, the secure multicast rate is well understood. Computing the secure multicast rate, or even the secure unicast rate, in the more general setting in which all network nodes may generate (independent) random keys is known to be as difficult as computing the (non-secure) capacity of multiple-unicast network coding instances --- a well known open problem. This work treats an …
In an unreliable single-hop broadcast network setting, we investigate the throughput and decoding-delay performance of random linear network coding as a function of the coding window size and the network size. Our model consists of a source transmitting packets of a single flow to a set of $n$ users over independent erasure channels. The source performs random linear network coding (RLNC) over $k$ (coding window size) packets and broadcasts them to the users. We note that the broadcast throughput of RLNC must vanish with increasing $n$, for any fixed $k.$ Hence, in contrast to other works in t…
We propose an efficient Adaptive Random Convolutional Network Coding (ARCNC) algorithm to address the issue of field size in random network coding. ARCNC operates as a convolutional code, with the coefficients of local encoding kernels chosen randomly over a small finite field. The lengths of local encoding kernels increase with time until the global encoding kernel matrices at related sink nodes all have full rank. Instead of estimating the necessary field size a priori, ARCNC operates in a small finite field. It adapts to unknown network topologies without prior knowledge, by locally increme…
In this work, we consider a distributed source coding problem with a joint distortion criterion depending on the sources and the reconstruction. This includes as a special case the problem of computing a function of the sources to within some distortion and also the classic Slepian-Wolf problem, Berger-Tung problem, Wyner-Ziv problem, Yeung-Berger problem and the Ahlswede-Korner-Wyner problem. While the prevalent trend in information theory has been to prove achievability results using Shannon's random coding arguments, using structured random codes offer rate gains over unstructured random co…
