In this paper, we consider encoding strategies for the Z-channel with noiseless feedback. We analyze the combinatorial setting where the maximum number of errors inflicted by an adversary is proportional to the number of transmissions, which goes to infinity. Without feedback, it is known that the rate of optimal asymmetric-error-correcting codes for the error fraction $τ\ge 1/4$ vanishes as the blocklength grows. In this paper, we give an efficient feedback encoding scheme with $n$ transmissions that achieves a positive rate for any fraction of errors $τ<1$ and $n\to\infty$. Additionally, we state an upper bound on the rate of asymptotically long feedback asymmetric error-correcting codes.
Noiseless coding theorem for finite stationary memoryless information sources is proved by using induction on the number of source symbols and the inequality of geometric and harmonic means.
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the domain of application is clarified. An examination of the bounds of the Theorem leads to a new symmetric restatement. It is shown that the extended upper bound is an acheivable upper bound, giving symmetry to the theorem.The relation of information entropy to the physical entropy of Gibbs and Boltmann is illustrated. Consequently, the study of Shannon Entropy i…
When designing a distributed control system, the system designer has a choice in how to connect the different units through communication channels. In practice, noiseless and noisy channels may coexist. Using the standard toy example of scalar stabilization, this paper shows how a small amount of noiseless feedback can perform a ``supervisory'' role and thereby boost the effectiveness of noisy feedback.
We consider the problem of communication over a two-user Additive White Gaussian Noise Broadcast Channel (AWGN-BC) with an AWGN Multiple Access (MAC) active feedback. We describe a constructive reduction from this setup to the well-studied setup of linear-feedback coding over the AWGN-BC with noiseless feedback (and different parameters). This reduction facilitates the design of linear-feedback coding schemes in the (passive) noiseless feedback regime, which can then be easily and constructively transformed into coding schemes in the MAC feedback regime that attain the exact same rates. Our co…
We consider the transmission of nonexponentially many messages through a binary symmetric channel with noiseless feedback. We obtain an upper bound for the best decoding error exponent. Combined with the corresponding known lower bound, this allows to find the reliability function for this channel at zero rate.
In this paper, first, we investigate the model of degraded broadcast channel with side information and confidential messages. This work is from Steinberg's work on the degraded broadcast channel with causal and noncausal side information, and Csisz$\acute{a}$r-Körner's work on broadcast channel with confidential messages. Inner and outer bounds on the capacity-equivocation regions are provided for the noncausal and causal cases. Superposition coding and double-binning technique are used in the corresponding achievability proofs. Then, we investigate the degraded broadcast channel with side inf…
The physical layer security in the up-link of the wireless communication systems is often modeled as the multiple access wiretap channel (MAC-WT), and recently it has received a lot attention. In this paper, the MAC-WT has been re-visited by considering the situation that the legitimate receiver feeds his received channel output back to the transmitters via two noiseless channels, respectively. This model is called the MAC-WT with noiseless feedback. Inner and outer bounds on the secrecy capacity region of this feedback model are provided. To be specific, we first present a decode-and-forward …
Berlekamp and Zigangirov completely determined the capacity error function for binary error correcting codes with noiseless feedback. It is still an unsolved problem if the upper bound for the capacity error function in the non-binary case of Ahlswede, Lebedev, and Deppe is sharp. We consider wraparound channels with limited magnitude and noiseless feedback. We completely determine the capacity error function for all $q$-ary wraparound channels with a magnitude of level $r$. All of our algorithms use partial noiseless feedback. Furthermore, a special case of the problem is equivalent to Shanno…
In this paper we develop a statistical mechanical interpretation of the noiseless source coding scheme based on an absolutely optimal instantaneous code. The notions in statistical mechanics such as statistical mechanical entropy, temperature, and thermal equilibrium are translated into the context of noiseless source coding. Especially, it is discovered that the temperature 1 corresponds to the average codeword length of an instantaneous code in this statistical mechanical interpretation of noiseless source coding scheme. This correspondence is also verified by the investigation using box-cou…
In this paper, we consider the problem of sequential transmission over the binary symmetric channel (BSC) with full, noiseless feedback. Naghshvar et al. proposed a one-phase encoding scheme, for which we refer to as the small-enough difference (SED) encoder, which can achieve capacity and Burnashev's optimal error exponent for symmetric binary-input channels. They also provided a non-asymptotic upper bound on the average blocklength, which implies an achievability bound on rates. However, their achievability bound is loose compared to the simulated performance of SED encoder, and even lies be…
We propose an information transmission scheme by a swarm of anonymous oblivious mobile robots on a graph. The swarm of robots travel from a sender vertex to a receiver vertex to transmit a symbol generated at the sender. The codeword for a symbol is a pair of an initial configuration at the sender and a set of terminal configurations at the receiver. The set of such codewords forms a code. We analyze the performance of the proposed scheme in terms of its code size and transmission delay. We first demonstrate that a lower bound of the transmission delay depends on the size of the swarm, and the…
In digital systems such as fiber optical communications, the ratio between probability of errors of type $1\to 0$ and $0 \to 1$ can be large. Practically, one can assume that only one type of error can occur. These errors arecalled asymmetric. Unidirectional errors differ from asymmetric type of errors; here both $1 \to 0$ and $0 \to 1$ type of errors are possible, but in any submittedcodeword all the errors are of the same type. This can be generalized for the $q$-ary case. We consider $q$-ary unidirectional channels with feedback and give bounds for the capacity error function. It turns out …
A new single-letter achievable rate region is proposed for the two-user discrete memoryless multiple-access channel(MAC) with noiseless feedback. The proposed region includes the Cover-Leung rate region [1], and it is shown that the inclusion is strict. The proof uses a block-Markov superposition strategy based on the observation that the messages of the two users are correlated given the feedback. The rates of transmission are too high for each encoder to decode the other's message directly using the feedback, so they transmit correlated information in the next block to learn the message of o…
Building on the work of Horstein, Shayevitz and Feder, and Naghshvar \emph{et al.}, this paper presents algorithms for low-complexity sequential transmission of a $k$-bit message over the binary symmetric channel (BSC) with full, noiseless feedback. To lower complexity, this paper shows that the initial $k$ binary transmissions can be sent before any feedback is required and groups messages with equal posteriors to reduce the number of posterior updates from exponential in $k$ to linear in $k$. Simulation results demonstrate that achievable rates for this full, noiseless feedback system approa…
Traditional channel coding with feedback constructs and transmits a codeword only after all message bits are available at the transmitter. This paper joins Guo & Kostina and Lalitha et. al. in developing approaches for causal (or progressive) encoding, where the transmitter may begin transmitting codeword symbols as soon as the first message bit arrives. Building on the work of Horstein, Shayevitz and Feder, and Naghshvar et. al., this paper extends our previous computationally efficient systematic algorithm for traditional posterior matching to produce a four-phase encoder that progressively …
In this paper, we consider the problem of evaluating capacity expression of a multiple access channel (MAC) with noiseless feedback. So far, the capacity expression for this channel is known through a multi letter directed information by Kramer [1]. Recently, it was shown in [2] that one can pose it as a dynamic optimization problem, however, no dynamic program was provided as the authors claimed there is no notion of state that is observed by both the senders. In this paper, we build upon [2] to show that there indeed exists a state and therefore a dynamic program (DP) that decomposes this dy…
A coding scheme for the discrete memoryless broadcast channel with {noiseless, noisy, generalized} feedback is proposed, and the associated achievable region derived. The scheme is based on a block-Markov strategy combining the Marton scheme and a lossy version of the Gray-Wyner scheme with side-information. In each block the transmitter sends fresh data and update information that allows the receivers to improve the channel outputs observed in the previous block. For a generalization of Dueck's broadcast channel our scheme achieves the noiseless-feedback capacity, which is strictly larger tha…
For a multi-source multi-terminal noiseless network, the key-dissemination problem involves the task of multicasting a secret key K from the network sources to its terminals. As in secure multicast network-coding, in the key-dissemination problem the source nodes have access to independent randomness and, as the network is noiseless, the resulting key K is a function of the sources' information. However, different from traditional forms of multicast, in key-dissemination the key K need not consist of source messages, but rather may be any function of the information generated at the sources, a…
Linear coding schemes have been the main choice of coding for the additive white Gaussian noise broadcast channel (AWGN-BC) with noiseless feedback in the literature. The achievable rate regions of these schemes go well beyond the capacity region of the AWGN-BC without feedback. In this paper, a concatenating coding design for the $K$-user AWGN-BC with noisy feedback is proposed that relies on linear feedback schemes to achieve rate tuples outside the no-feedback capacity region. Specifically, a linear feedback code for the AWGN-BC with noisy feedback is used as an inner code that creates an e…
We consider networks of noisy degraded wiretap channels in the presence of an eavesdropper. For the case where the eavesdropper can wiretap at most one channel at a time, we show that the secrecy capacity region, for a broad class of channels and any given network topology and communication demands, is equivalent to that of a corresponding network where each noisy wiretap channel is replaced by a noiseless wiretap channel. Thus in this case there is a separation between wiretap channel coding on each channel and secure network coding on the resulting noiseless network. We show with an example …
In the setting of error-correcting codes with feedback, Alice wishes to communicate a $k$-bit message $x$ to Bob by sending a sequence of bits over a channel while noiselessly receiving feedback from Bob. It has been long known (Berlekamp, 1964) that in this model, Bob can still correctly determine $x$ even if $\approx \frac13$ of Alice's bits are flipped adversarially. This improves upon the classical setting without feedback, where recovery is not possible for error fractions exceeding $\frac14$. The original feedback setting assumes that after transmitting each bit, Alice knows (via feedbac…
An index coding problem arises when there is a single source with a number of messages and multiple receivers each wanting a subset of messages and knowing a different set of messages a priori. The noiseless Index Coding Problem is to identify the minimum number of transmissions (optimal length) to be made by the source through noiseless channels so that all receivers can decode their wanted messages using the transmitted symbols and their respective prior information. Recently, it is shown that different optimal length codes perform differently in a noisy channel. Towards identifying the best…
We consider computations over networks with multiple broadcast channels that intersect at a single party. Each broadcast link suffers from random bit-flip noise that affects the receivers independently. We design interactive coding schemes that successfully perform any computation over these noisy networks and strive to reduce their communication overhead with respect to the original (noiseless) computation. A simple variant of a coding scheme by Rajagopalan and Schulman (STOC 1994) shows that any (noiseless) protocol of $R$ rounds can be reliably simulated in $O(R\log n)$ rounds over a networ…
We introduce a novel mechanism, called timid/bold coding, by which feedback can be used to improve coding performance. For a certain class of DMCs, called compound-dispersion channels, we show that timid/bold coding allows for an improved second-order coding rate compared with coding without feedback. For DMCs that are not compound dispersion, we show that feedback does not improve the second-order coding rate. Thus we completely determine the class of DMCs for which feedback improves the second-order coding rate. An upper bound on the second-order coding rate is provided for compound-dispersi…
This paper proves the separation between source-network coding and channel coding in networks of noisy, discrete, memoryless channels. We show that the set of achievable distortion matrices in delivering a family of dependent sources across such a network equals the set of achievable distortion matrices for delivering the same sources across a distinct network which is built by replacing each channel by a noiseless, point-to-point bit-pipe of the corresponding capacity. Thus a code that applies source-network coding across links that are made almost lossless through the application of independ…
Message Authentication Code (MAC) is a keyed function $f_K$ such that when Alice, who shares the secret $K$ with Bob, sends $f_K(M)$ to the latter, Bob will be assured of the integrity and authenticity of $M$. Traditionally, it is assumed that the channel is noiseless. However, Maurer showed that in this case an attacker can succeed with probability $2^{-\frac{H(K)}{\ell+1}}$ after authenticating $\ell$ messages. In this paper, we consider the setting where the channel is noisy. Specifically, Alice and Bob are connected by a discrete memoryless channel (DMC) $W_1$ and a noiseless but insecure …
We initiate the study of zero-error communication via quantum channels when the receiver and sender have at their disposal a noiseless feedback channel of unlimited quantum capacity, generalizing Shannon's zero-error communication theory with instantaneous feedback. We first show that this capacity is a function only of the linear span of Choi-Kraus operators of the channel, which generalizes the bipartite equivocation graph of a classical channel, and which we dub "non-commutative bipartite graph". Then we go on to show that the feedback-assisted capacity is non-zero (with constant activating…
Cryptographic protocols are often implemented at upper layers of communication networks, while error-correcting codes are employed at the physical layer. In this paper, we consider utilizing readily-available physical layer functions, such as encoders and decoders, together with shared keys to provide a threshold-type security scheme. To this end, we first consider a scenario where the effect of the physical layer is omitted and all the channels between the involved parties are assumed to be noiseless. We introduce a model for threshold-secure coding, where the legitimate parties communicate u…
In this paper, we investigate the additive Gaussian noise channel with noisy feedback. We consider the setup of linear coding of the feedback information and Gaussian signaling of the message (i.e. Cover-Pombra Scheme). Then, we derive the upper and lower bounds on the largest achievable rate for this setup. We show that these two bounds can be obtained by solving two convex optimization problems. Finally, we present some simulations and discussion.
Secret key establishment leveraging the physical layer as a source of common randomness has been investigated in a range of settings. We investigate the problem of establishing, in an information-theoretic sense, a secret key between a user and a base-station (BS) (more generally, part of a wireless infrastructure), but for two such user-BS pairs attempting the key establishment simultaneously. The challenge in this novel setting lies in that a user can eavesdrop another BS-user communications. It is thus paramount to ensure the two keys are established with no leakage to the other user, in sp…
Adder MAC is a simple noiseless multiple-access channel (MAC), where if users send messages $X_1,\ldots,X_h\in \{0,1\}^n$, then the receiver receives $Y = X_1+\cdots+X_h$ with addition over $\mathbb{Z}$. Communication over the noiseless adder MAC has been studied for more than fifty years. There are two models of particular interest: uniquely decodable code tuples, and $B_h$-codes. In spite of the similarities between these two models, lower bounds and upper bounds of the optimal sum rate of uniquely decodable code tuple asymptotically match as number of users goes to infinity, while there is …
Optimal coding over the additive white Gaussian noise channel under the peak energy constraint is studied when there is noisy feedback over an orthogonal additive white Gaussian noise channel. As shown by Pinsker, under the peak energy constraint, the best error exponent for communicating an M-ary message, M >= 3, with noise-free feedback is strictly larger than the one without feedback. This paper extends Pinsker's result and shows that if the noise power in the feedback link is sufficiently small, the best error exponent for conmmunicating an M-ary message can be strictly larger than the one…
We consider the problem of zero error source coding with limited feedback when side information is present at the receiver. First, we derive an achievable rate region for arbitrary joint distributions on the source and the side information. When all source pairs of source and side information symbols are observable with non-zero probability, we show that this characterization gives the entire rate region. Next, we demonstrate a class of sources for which asymptotically zero feedback suffices to achieve zero-error coding at the rate promised by the Slepian-Wolf bound for asymptotically lossless…
The model of wiretap channel (WTC) is important as it constitutes the essence of physical layer security (PLS). Wiretap channel with noiseless feedback (WTC-NF) is especially interesting as it shows what can be done when a private feedback is available. The already existing secret key based feedback coding scheme focuses on generating key from the feedback and using this key to protect part of the transmitted message. It has been shown that this secret key based feedback coding scheme is only optimal for the degraded WTC-NF, and finding an optimal feedback scheme for the general WTC-NF motivat…
We show that for the multi-antenna Gaussian multi-access channel (MAC) and broadcast channel (BC) with perfect feedback, the rate regions achieved by linear-feedback coding schemes (called linear-feedback capacity regions) coincide when the same total input-power constraint is imposed on both channels and when the MAC channel matrices are the transposes of the BC channel matrices. Such a pair of MAC and BC is called dual. We also identify sub-classes of linear-feedback coding schemes that achieve the linear-feedback capacity regions of these two channels and present multi-letter expressions fo…
Coding schemes for discrete memoryless multicast networks (DM-MN) with rate-limited feedback from the receivers and relays to the transmitter are proposed. The schemes improve over the noisy network coding proposed by Lim et al.. For the single relay channel with relay-transmitter feedback, our coding schemes recover Gabbai and Bross's results, and strictly improve on noisy network coding, distributed decode-forward coding proposed by Lim et al., and all known lower bounds on the achievable rates proposed for the setup without feedback. The coding schemes are based on block-Markov coding, supe…
The reliability function of memoryless channels with noiseless feedback and variable-length coding has been found to be a linear function of the average rate in the classic work of Burnashev. In this work we consider unifilar channels with noiseless feedback and study upper bounds for the channel reliability function with variable length codes. In unifilar channels the channel state is known to the transmitter but is unknown to the receiver. We generalize Burnashev's analysis and derive a similar expression which is linear in average rate and depends on the channel capacity, as well as an addi…
This paper considers the memoryless input-constrained binary erasure channel (BEC). The channel input constraint is the $(d,\infty)$-runlength limited (RLL) constraint, which mandates that any pair of successive $1$s in the input sequence be separated by at least $d$ $0$s. We consider a scenario where there is causal, noiseless feedback from the decoder. We demonstrate a simple, labelling-based, zero-error feedback coding scheme, which we prove to be feedback capacity-achieving, and, as a by-product, obtain an explicit characterization of the feedback capacity. Our proof is based on showing th…
We consider the problem of compression of two memoryless binary sources, the correlation between which is defined by a Hidden Markov Model (HMM). We propose a Decision Feedback (DF) based scheme which when used with low density parity check codes results in compression close to the Slepian Wolf limits.
We consider the problem of compression of two memoryless binary sources, the correlation between which is defined by a Hidden Markov Model (HMM). We propose a Decision Feedback (DF) based scheme which when used with low density parity check codes results in compression close to the Slepian Wolf limits.
We consider the problem of source coding with receiver side information for the simple network proposed by R. Gray and A. Wyner in 1974. In this network, a transmitter must reliably transport the output of two correlated information sources to two receivers using three noiseless channels: a public channel which connects the transmitter to both receivers, and two private channels which connect the transmitter directly to each receiver. We extend Gray and Wyner's original problem by permitting side information to be present at each receiver. We derive inner and outer bounds for the achievable ra…
A coding scheme is proposed for the memoryless Gaussian broadcast channel with correlated noises and feedback. For all noise correlations other than -1, the gap between the sum-rate the scheme achieves and the full-cooperation bound vanishes as the signal-to-noise ratio tends to infinity. When the correlation coefficient is -1, the gains afforded by feedback are unbounded and the prelog is doubled. When the correlation coefficient is +1 we demonstrate a dichotomy: If the noise variances are equal, then feedback is useless, and otherwise, feedback affords unbounded rate gains and doubles the pr…
In this paper, we propose capacity-achieving communication schemes for Gaussian finite-state Markov channels (FSMCs) subject to an average channel input power constraint, under the assumption that the transmitters can have access to delayed noiseless output feedback as well as instantaneous or delayed channel state information (CSI). We show that the proposed schemes reveals connections between feedback communication and feedback control.
We consider the problem of rate and power allocation for a sensor network under the pairwise distributed source coding constraint. For noiseless source-terminal channels, we show that the minimum sum rate assignment can be found by finding a minimum weight arborescence in an appropriately defined directed graph. For orthogonal noisy source-terminal channels, the minimum sum power allocation can be found by finding a minimum weight matching forest in a mixed graph. Numerical results are presented for both cases showing that our solutions always outperform previously proposed solutions. The gain…
The Ising channel, which was introduced in 1990, is a channel with memory that models Inter-Symbol interference. In this paper we consider the Ising channel with feedback and find the capacity of the channel together with a capacity-achieving coding scheme. To calculate the channel capacity, an equivalent dynamic programming (DP) problem is formulated and solved. Using the DP solution, we establish that the feedback capacity is the expression $C=(\frac{2H_b(a)}{3+a})\approx 0.575522$ where $a$ is a particular root of a fourth-degree polynomial and $H_b(x)$ denotes the binary entropy function. …
This paper quantifies the fundamental limits of variable-length transmission of a general (possibly analog) source over a memoryless channel with noiseless feedback, under a distortion constraint. We consider excess distortion, average distortion and guaranteed distortion ($d$-semifaithful codes). In contrast to the asymptotic fundamental limit, a general conclusion is that allowing variable-length codes and feedback leads to a sizable improvement in the fundamental delay-distortion tradeoff. In addition, we investigate the minimum energy required to reproduce $k$ source samples with a given f…
The error exponent of Markov channels with feedback is studied in the variable-length block-coding setting. Burnashev's classic result is extended and a single letter characterization for the reliability function of finite-state Markov channels is presented, under the assumption that the channel state is causally observed both at the transmitter and at the receiver side. Tools from stochastic control theory are used in order to treat channels with intersymbol interference. In particular the convex analytical approach to Markov decision processes is adopted to handle problems with stopping time…
Distributed source coding is traditionally viewed in the block coding context -- all the source symbols are known in advance at the encoders. This paper instead considers a streaming setting in which iid source symbol pairs are revealed to the separate encoders in real time and need to be reconstructed at the decoder with some tolerable end-to-end delay using finite rate noiseless channels. A sequential random binning argument is used to derive a lower bound on the error exponent with delay and show that both ML decoding and universal decoding achieve the same positive error exponents inside t…
We consider the problem of zero-delay coding of a dynamical system over a discrete noiseless channel under three estimation criteria concerned with the low-distortion regime. For these three criteria, formulated stochastically in terms of a probability distribution for the initial state, we characterize the smallest channel capacities above which the estimation objectives can be achieved. The results establish further connections between topological and metric entropy of dynamical systems and information theory.
Achievable rate regions for cooperative relay broadcast channels with rate-limited feedback are proposed. Specifically, we consider two-receiver memoryless broadcast channels where each receiver sends feedback signals to the transmitter through a noiseless and rate-limited feedback link, and meanwhile, acts as relay to transmit cooperative information to the other receiver. It's shown that the proposed rate regions improve on the known regions that consider either relaying cooperation or feedback communication, but not both.
Most communication systems use some form of feedback, often related to channel state information. The common models used in analyses either assume perfect channel state information at the receiver and/or noiseless state feedback links. However, in practical systems, neither is the channel estimate known perfectly at the receiver and nor is the feedback link perfect. In this paper, we study the achievable diversity multiplexing tradeoff using i.i.d. Gaussian codebooks, considering the errors in training the receiver and the errors in the feedback link for FDD systems, where the forward and the …
