The capacity of a time-varying block-memoryless channel in which the transmitter and the receiver have access to (possibly different) noisy causal channel side information (CSI) is obtained. It is shown that the capacity formula obtained in this correspondence reduces to the capacity formula reported in \cite{Gold07} for the special case where the transmitter CSI is a deterministic function of the receiver CSI.
In this paper, we examine discrete memoryless Multiple Access Channels (MACs) with two-sided feedback in the presence of two correlated channel states that are correlated in the sense of Slepian-Wolf (SW). We find achievable rate region for this channel when the states are provided non-causally to the transmitters and show that our achievable rate region subsumes Cover-Leung achievable rate for the discrete memoryless MAC with two-sided feedback as its special case. We also find the capacity region of discrete memoryless MAC with two-sided feedback and with SW-type correlated states available …
The sum capacity of a class of discrete memoryless interference channels is determined. This class of channels is defined analogous to the Gaussian Z-interference channel with weak interference; as a result, the sum capacity is achieved by letting the transceiver pair subject to the interference communicates at a rate such that its message can be decoded at the unintended receiver using single user detection. Moreover, this class of discrete memoryless interference channels is equivalent in capacity region to certain discrete degraded interference channels. This allows the construction of a ca…
We consider the two-receiver memoryless broadcast channel with states where each receiver requests both common and private messages, and may know part of the private message requested by the other receiver as receiver message side information (RMSI). We address two categories of the channel (i) channel with states known causally to the transmitter, and (ii) channel with states known non-causally to the transmitter. Starting with the channel without RMSI, we first propose a transmission scheme and derive an inner bound for the causal category. We then unify our inner bound for the causal catego…
We propose a new inner bound on the capacity region of a memoryless multiple-access channel that is governed by a memoryless state that is known strictly causally to the encoders. The new inner bound contains the previous bounds, and we provide an example demonstrating that the inclusion can be strict. A variation on this example is then applied to the case where the channel is governed by two independent state sequences, where each transmitter knows one of the states strictly causally. The example proves that, as conjectured by Li et al., an inner bound that they derived for this scenario can…
In this paper, we study the state-dependent two-user interference channel, where the state information is non-causally known at both transmitters but unknown to either of the receivers. We propose two coding schemes for the discrete memoryless case: simultaneous encoding for the sub-messages in the first one and superposition encoding in the second one, both with rate splitting and Gel'fand-Pinsker coding. The corresponding achievable rate regions are established.
We consider the capacity of memoryless finite-state multiple access channel (FS-MAC) with causal asymmetric noisy state information available at both transmitters and complete state information available at the receiver. Single letter inner and outer bounds are provided for the capacity of such channels when the state process is independent and identically distributed. The outer bound is attained by observing that the proposed inner bound is tight for the sum-rate capacity.
We propose a method to study the secrecy constraints in key generation problems where side information might be present at untrusted users. Our method is inspired by a recent work of Hayashi and Tan who used the Rényi divergence as the secrecy measure to study the output statistics of applying hash functions to a random sequence. By generalizing the achievability result of Hayashi and Tan to the multi-terminal case, we obtain the output statistics of applying hash functions to multiple random sequences, which turn out to be an important tool in the achievability proof of strong secrecy capacit…
The problem of state communication over a discrete memoryless channel with discrete memoryless state is studied when the state information is available strictly causally at the encoder. It is shown that block Markov encoding, in which the encoder communicates a description of the state sequence in the previous block by incorporating side information about the state sequence at the decoder, yields the minimum state estimation error. When the same channel is used to send additional independent information at the expense of a higher channel state estimation error, the optimal tradeoff between the…
We study communication systems over state-dependent channels in the presence of a malicious state-aware jamming adversary. The channel has a memoryless state with an underlying distribution. The adversary introduces a jamming signal into the channel. The state sequence is known non-causally to both the encoder and the adversary. Taking an Arbitrarily Varying Channel (AVC) approach, we consider two setups, namely, the discrete memoryless Gel'fand-Pinsker (GP) AVC and the additive white Gaussian Dirty Paper (DP) AVC. We determine the randomized coding capacity of both the AVCs under a maximum pr…
Coding for the causal cognitive radio channel, with the cognitive source subjected to a half-duplex constraint, is studied. A discrete memoryless channel model incorporating the half-duplex constraint is presented, and a new achievable rate region is derived for this channel. It is proved that this rate region contains the previously known causal achievable rate region of \cite{Devroye06} for Gaussian channels.
The source-coding problem with side information at the decoder is studied subject to a constraint that the encoder---to whom the side information is unavailable---be able to compute the decoder's reconstruction sequence to within some distortion. For discrete memoryless sources and finite single-letter distortion measures, an expression is given for the minimal description rate as a function of the joint law of the source and side information and of the allowed distortions at the encoder and at the decoder. The minimal description rate is also computed for a memoryless Gaussian source with squ…
The problem of distributed binary hypothesis testing in the Gray-Wyner network with side information is studied in this paper. An observer has access to a discrete memoryless and stationary source and describes its observation to two detectors via one common and two private channels. The channels are considered error-free but rate-limited. Each detector also has access to its own discrete memoryless and stationary source, i.e., the side information. The goal is to perform two distinct binary hypothesis testings on the joint distribution of observations at detectors. Additionally, the observer …
A communication setup is considered where a transmitter wishes to simultaneously sense its channel state and convey a message to a receiver. The state is estimated at the transmitter by means of generalized feedback, i.e. a strictly causal channel output that is observed at the transmitter. The scenario is motivated by a joint radar and communication system where the radar and data applications share the same frequency band. For the case of a memoryless channel with i.i.d. state sequences, we characterize the capacity-distortion tradeoff, defined as the best achievable rate below which a messa…
In this work we present results about the rate of (relative) information loss induced by passing a real-valued, stationary stochastic process through a memoryless system. We show that for a special class of systems the information loss rate is closely related to the difference of differential entropy rates of the input and output processes. It is further shown that the rate of (relative) information loss is bounded from above by the (relative) information loss the system induces on a random variable distributed according to the process's marginal distribution. As a side result, in this work we…
This paper studies the capacity of a class of discrete memoryless interference channels where interference is defined analogous to that of Gaussian interference channel with one-sided weak interference. The sum-rate capacity of this class of channels is determined. As with the Gaussian case, the sum-rate capacity is achieved by letting the transceiver pair subject to interference communicate at a rate such that its message can be decoded at the unintended receiver using single user detection. It is also established that this class of discrete memoryless interference channels is equivalent in c…
We address the problem of universal communications over an unknown channel with an instantaneous noiseless feedback, and show how rates corresponding to the empirical behavior of the channel can be attained, although no rate can be guaranteed in advance. First, we consider a discrete modulo-additive channel with alphabet $\mathcal{X}$, where the noise sequence $Z^n$ is arbitrary and unknown and may causally depend on the transmitted and received sequences and on the encoder's message, possibly in an adversarial fashion. Although the classical capacity of this channel is zero, we show that rate…
The problem of lossless fixed-rate streaming coding of discrete memoryless sources with side information at the decoder is studied. A random time-varying tree-code is used to sequentially bin strings and a Stack Algorithm with a variable bias uses the side information to give a delay-universal coding system for lossless source coding with side information. The scheme is shown to give exponentially decaying probability of error with delay, with exponent equal to Gallager's random coding exponent for sources with side information. The mean of the random variable of computation for the stack deco…
The problem of lossless data compression with side information available to both the encoder and the decoder is considered. The finite-blocklength fundamental limits of the best achievable performance are defined, in two different versions of the problem: Reference-based compression, when a single side information string is used repeatedly in compressing different source messages, and pair-based compression, where a different side information string is used for each source message. General achievability and converse theorems are established for arbitrary source-side information pairs. Nonasymp…
The interference channel models a wireless network where several source-destination pairs compete for the same resources. When nodes transmit simultaneously the destinations experience interference. This paper considers a 4-node network, where two nodes are sources and the other two are destinations. All nodes are full-duplex and cooperate to mitigate interference. A sum-rate outer bound is derived, which is shown to unify a number of previously derived outer bounds for special cases of cooperation. The approach is shown to extend to cooperative interference networks with more than two source-…
A coding scheme for the discrete memoryless broadcast channel with {noiseless, noisy, generalized} feedback is proposed, and the associated achievable region derived. The scheme is based on a block-Markov strategy combining the Marton scheme and a lossy version of the Gray-Wyner scheme with side-information. In each block the transmitter sends fresh data and update information that allows the receivers to improve the channel outputs observed in the previous block. For a generalization of Dueck's broadcast channel our scheme achieves the noiseless-feedback capacity, which is strictly larger tha…
We consider source coding with fixed lag side information at the decoder. We focus on the special case of perfect side information with unit lag corresponding to source coding with feedforward (the dual of channel coding with feedback) introduced by Pradhan. We use this duality to develop a linear complexity algorithm which achieves the rate-distortion bound for any memoryless finite alphabet source and distortion measure.
We investigate two source coding problems with secrecy constraints. In the first problem we consider real--time fully secure transmission of a memoryless source. We show that although classical variable--rate coding is not an option since the lengths of the codewords leak information on the source, the key rate can be as low as the average Huffman codeword length of the source. In the second problem we consider causal source coding with a fidelity criterion and side information at the decoder and the eavesdropper. We show that when the eavesdropper has degraded side information, it is optimal …
In this paper we provide an achievable rate region for the discrete memoryless multiple access channel with correlated state information known non-causally at the encoders using a random binning technique. This result is a generalization of the random binning technique used by Gel'fand and Pinsker for the problem with non-causal channel state information at the encoder in point to point communication.
We identify the common underlying form of the capacity expression that is applicable to both cases where causal or non-causal side information is made available to the transmitter. Using this common form we find that for the single user channel, the multiple access channel, the degraded broadcast channel, and the degraded relay channel, the sum capacity with causal and non-causal side information are identical when all the transmitter side information is also made available to all the receivers. A genie-aided outerbound is developed that states that when a genie provides $n$ bits of side infor…
We consider a two-user state-dependent multiaccess channel in which the states of the channel are known non-causally to one of the encoders and only strictly causally to the other encoder. Both encoders transmit a common message and, in addition, the encoder that knows the states non-causally transmits an individual message. We find explicit characterizations of the capacity region of this communication model in both discrete memoryless (DM) and memoryless Gaussian cases. In particular the capacity region analysis demonstrates the utility of the knowledge of the states only strictly causally a…
We propose a novel tensor-based formalism for inferring causal structures from time series. An information theoretical analysis of transfer entropy, shows that transfer entropy results from transmission of information over a set of communication channels. Tensors are the mathematical equivalents of these multi-channel causal channels. A multi-channel causal channel is a generalization of a discrete memoryless channel. Investigation of a system comprising three variables shows that in our formalism, bivariate analysis suffices to differentiate between direct and indirect relations. For this to …
In this paper, we study causal discrete-memoryless relay networks (DMRNs). The network consists of multiple nodes, each of which can be a source, relay, and/or destination. In the network, there are two types of relays, i.e., relays with one sample delay (strictly causal) and relays without delay (causal) whose transmit signal depends not only on the past received symbols but also on the current received symbol. For this network, we derive two new cut-set bounds, one when the causal relays have their own messages and the other when not. Using examples of a causal vector Gaussian two-way relay …
This paper studies the problem of secure communcation over the two-receiver discrete memoryless broadcast channel with one-sided receiver side information and with a passive eavesdropper. We proposed a coding scheme which is based upon the superposition-Marton framework. Secrecy techniques such as the one-time pad, Carleial-Hellman secrecy coding and Wyner serecy coding are applied to ensure individual secrecy. This scheme is shown to be capacity achieving for some cases of the degraded broadcast channel. We also notice that one-sided receiver side information provides the advantage of rate re…
Secure communication in a potentially malicious environment becomes more and more important. The arbitrarily varying wiretap channel (AVWC) provides information theoretical bounds on how much information can be exchanged even in the presence of an active attacker. If the active attacker has non-causal side information, situations in which a legitimate communication system has been hacked, can be modeled. We investigate the AVWC with non-causal side information at the jammer for the case that there exists a best channel to the eavesdropper. Non-causal side information means that the transmitted…
In this work, we establish a full single-letter characterization of the rate-distortion region of an instance of the Gray-Wyner model with side information at the decoders. Specifically, in this model an encoder observes a pair of memoryless, arbitrarily correlated, sources $(S^n_1,S^n_2)$ and communicates with two receivers over an error-free rate-limited link of capacity $R_0$, as well as error-free rate-limited individual links of capacities $R_1$ to the first receiver and $R_2$ to the second receiver. Both receivers reproduce the source component $S^n_2$ losslessly; and Receiver $1$ also r…
We investigate the combination between causal/zero-delay source coding and information-theoretic secrecy. Two source coding models with secrecy constraints are considered. We start by considering zero-delay perfectly secret lossless transmission of a memoryless source. We derive bounds on the key rate and coding rate needed for perfect zero-delay secrecy. In this setting, we consider two models which differ by the ability of the eavesdropper to parse the bit-stream passing from the encoder to the legitimate decoder into separate messages. We also consider causal source coding with a fidelity c…
In this paper, we consider a more general four-terminal memoryless relay-eavesdropper channel with state information (REC-SI) and derive an achievable perfect secrecy rate for it. We suppose that the state information is non-causally available at the transmitter and relay only. The transmitter wishes to establish a secure communication with the legitimate receiver by the help of a relay where a confidential message will be kept secret from a passive eavesdropper. We consider active cooperation between the relay and transmitter. The relay helps the transmitter by relaying the message using deco…
In this paper, we introduce a discrete memoryless State-Dependent Relay Channel with Private Messages (SD-RCPM) as a generalization of the state-dependent relay channel. We investigate two main cases: SD-RCPM with non-causal Channel State Information (CSI), and SD-RCPM with causal CSI. In each case, it is assumed that partial CSI is available at the source and relay. For non-causal case, we establish an achievable rate region using Gel'fand-Pinsker type coding scheme at the nodes informed of CSI, and Compress-and-Forward (CF) scheme at the relay. Using Shannon's strategy and CF scheme, an achi…
We consider the $k$-user successive refinement problem with causal decoder side information and derive an exponential strong converse theorem. The rate-distortion region for the problem can be derived as a straightforward extension of the two-user case by Maor and Merhav (2008). We show that for any rate-distortion tuple outside the rate-distortion region of the $k$-user successive refinement problem with causal decoder side information, the joint excess-distortion probability approaches one exponentially fast. Our proof follows by judiciously adapting the recently proposed strong converse tec…
Polar coding over a class of binary discrete memoryless channels with channel knowledge at the encoder is studied. It is shown that polar codes achieve the capacity of convex and one-sided classes of symmetric channels.
The recently proposed set-up of source coding with a side information "vending machine" allows the decoder to select actions in order to control the quality of the side information. The actions can depend on the message received from the encoder and on the previously measured samples of the side information, and are cost constrained. Moreover, the final estimate of the source by the decoder is a function of the encoder's message and depends causally on the side information sequence. Previous work by Permuter and Weissman has characterized the rate-distortion-cost function in the special case i…
The cognitive interference channel is an interference channel in which one transmitter is non-causally provided with the message of the other transmitter. This channel model has been extensively studied in the past years and capacity results for certain classes of channels have been proved. In this paper we present new inner and outer bounds for the capacity region of the cognitive interference channel as well as new capacity results. Previously proposed outer bounds are expressed in terms of auxiliary random variables for which no cardinality constraint is known. Consequently it is not possib…
The two-user Multiple Access Channel (MAC) with cooperative encoders and Channel State Information (CSI) is considered where two different scenarios are investigated: A two-user MAC with common message (MACCM) and a two-user MAC with conferencing encoders (MACCE). For both situations, the two cases where the CSI is known to the encoders either non-causally or causally are studied. Achievable rate regions are established for both discrete memoryless channels and Gaussian channels with additive interference. The achievable rate regions derived for the Gaussian models with additive interference k…
We consider zero-delay single-user and multi-user source coding with average distortion constraint and decoder side information. The zero-delay constraint translates into causal (sequential) encoder and decoder pairs as well as the use of instantaneous codes. For the single-user setting, we show that optimal performance is attained by time sharing at most two scalar encoder-decoder pairs, that use zero-error side information codes. Side information lookahead is shown to useless in this setting. We show that the restriction to causal encoding functions is the one that causes the performance deg…
This paper simplifies an existing coding scheme for the two-receiver discrete memoryless broadcast channel with complementary receiver side information where there is a passive eavesdropper and individual secrecy is required. The existing coding scheme is simplified in two steps by replacing Wyner secrecy coding with Carleial-Hellman secrecy coding. The resulting simplified scheme is free from redundant message splits and random components. Not least, the simplified scheme retains the existing achievable individual secrecy rate region. Finally, its construction simplicity helps us gain additio…
We propose two coding schemes for the two-receiver discrete memoryless broadcast channel (BC) with rate-limited feedback from one or both receivers. They improve over the nofeedback capacity region for a large class of channels, including the class of \emph{strictly essentially less-noisy BCs} that we introduce in this article. Examples of strictly essentially less-noisy BCs are the binary symmetric BC (BSBC) or the binary erasure BC (BEBC) with unequal cross-over or erasure probabilities at the two receivers. When the feedback rates are sufficiently large, our schemes recover all previously k…
In this work, we study two models of arbitrarily varying channels, when causal side information is available at the encoder in a causal manner. First, we study the arbitrarily varying channel (AVC) with input and state constraints, when the encoder has state information in a causal manner. Lower and upper bounds on the random code capacity are developed. A lower bound on the deterministic code capacity is established in the case of a message-averaged input constraint. In the setting where a state constraint is imposed on the jammer, while the user is under no constraints, the random code bound…
We study the problem of coding over a general discrete memoryless broadcast channel controlled by random parameters. The parameters are available at the transmitter in a non-causal manner and are subject to a state masking constraint on the receivers. We derive inner and outer bounds on the achievable region and show that for the special case of Gaussian broadcast channel with private messages, these bounds are tight.
In this work, we consider a partially cooperative relay broadcast channel (PC-RBC) controlled by random parameters. We provide rate regions for two different situations: 1) when side information (SI) S^n on the random parameters is non-causally known at both the source and the relay and, 2) when side information S^n is non-causally known at the source only. These achievable regions are derived for the general discrete memoryless case first and then extended to the case when the channel is degraded Gaussian and the SI is additive i.i.d. Gaussian. In this case, the source uses generalized dirty …
A fading broadcast channel is considered where the transmitter employs two antennas and each of the two receivers employs a single receive antenna. It is demonstrated that even if the realization of the fading is precisely known to the receivers, the high signal-to-noise (SNR) throughput is greatly reduced if, rather than knowing the fading realization \emph{precisely}, the trasmitter only knows the fading realization \emph{approximately}. The results are general and are not limited to memoryless Gaussian fading.
In this work, we study the arbitrarily varying broadcast channel (AVBC), when state information is available at the transmitter in a causal manner. We establish inner and outer bounds on both the random code capacity region and the deterministic code capacity region with degraded message sets. The capacity region is then determined for a class of channels satisfying a condition on the mutual informations between the strategy variables and the channel outputs. As an example, we consider the arbitrarily varying binary symmetric broadcast channel with correlated noises. We show cases where the co…
We establish the rate region of an extended Gray-Wyner system for 2-DMS $(X,Y)$ with two additional decoders having complementary causal side information. This extension is interesting because in addition to the operationally significant extreme points of the Gray-Wyner rate region, which include Wyner's common information, G{á}cs-K{ö}rner common information and information bottleneck, the rate region for the extended system also includes the K{ö}rner graph entropy, the privacy funnel and excess functional information, as well as three new quantities of potential interest, as extreme points. T…
The fact that the results for 2-receiver broadcast channels (BCs) are not generalized to the 3-receiver ones is of information theoretical importance. In this paper we study two classes of discrete memoryless BCs with non-causal side information (SI), i.e. multilevel BC (MBC) and 3-receiver less noisy BC. First, we obtain an achievable rate region and a capacity outer bound for the MBC. Second, we prove a special capacity region for the 3-receiver less noisy BC. Third, the obtained special capacity region for the 3-receiver less noisy BC is extended to continuous alphabet fading Gaussian versi…
We study the problem of communication over a discrete arbitrarily varying channel (AVC) when a noisy version of the state is known non-causally at the encoder. The state is chosen by an adversary which knows the coding scheme. A state-myopic encoder observes this state non-causally, though imperfectly, through a noisy discrete memoryless channel (DMC). We first characterize the capacity of this state-dependent channel when the encoder-decoder share randomness unknown to the adversary, i.e., the randomized coding capacity. Next, we show that when only the encoder is allowed to randomize, the ca…
We consider the problem of causal source coding and causal decoding of a Gauss--Markov source, where the decoder has causal access to a side-information signal. We define the information causal rate-distortion function with causal decoder side information and prove that it bounds from below its operational counterpart. We further explain how to adapt the result to the setting of control over communication channels.
A computable expression for the rate-distortion (RD) function proposed by Heegard and Berger has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for \emph{physically degraded} side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete memoryless sources). In this paper, we consider a new setup in which the side information at one receiver is \emph{conditionally less noisy} than the side information at the other. The new setup includes…
