The interference channel is the simplest communication scenario where multiple autonomous users compete for shared resources. We combine game theory and information theory to define a notion of a Nash equilibrium region of the interference channel. The notion is game theoretic: it captures the selfish behavior of each user as they compete. The notion is also information theoretic: it allows each user to use arbitrary communication strategies as it optimizes its own performance. We give an exact characterization of the Nash equilibrium region of the two-user linear deterministic interference channel and an approximate characterization of the Nash equilibrium region of the two-user Gaussian interference channel to within 1 bit/s/Hz..
We discuss why MMSE estimation arises in lattice-based schemes for approaching the capacity of linear Gaussian channels, and comment on its properties.
We continue to discuss why MMSE estimation arises in coding schemes that approach the capacity of linear Gaussian channels. Here we consider schemes that involve successive decoding, such as decision-feedback equalization or successive cancellation.
In this contribution, the Gaussian channel is represented as an equivalent thermal system allowing to express its input-output mutual information in terms of thermodynamic quantities. This thermodynamic description of the mutual information is based upon a generalization of the $2^{nd}$ thermodynamic law and provides an alternative proof to the Guo-Shamai-Verdú theorem, giving an intriguing connection between this remarkable theorem and the most fundamental laws of nature - the laws of thermodynamics.
The Gray-Wyner network subject to a fidelity criterion is studied. Upper and lower bounds for the trade-offs between the private sum-rate and the common rate are obtained for arbitrary sources subject to mean-squared error distortion. The bounds meet exactly, leading to the computation of the rate region, when the source is jointly Gaussian. They meet partially when the sources are modeled via an additive Gaussian "channel". The bounds are inspired from the Shannon bounds on the rate-distortion problem.
We study a setting where a receiver must design a questionnaire to recover a sequence of symbols known to strategic sender, whose utility may not be incentive compatible. We allow the receiver the possibility of selecting the alternatives presented in the questionnaire, and thereby linking decisions across the components of the sequence. We show that, despite the strategic sender and the noise in the channel, the receiver can recover exponentially many sequences, but also that exponentially many sequences are unrecoverable even by the best strategy. We define the growth rate of the number of r…
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear transformations between their probability mass functions. The resulting tensor formalism is used to show that in specific cases, e.g., in the case the system consists of three stochastic processes, bivariate analysis suffices to distinguish true relations from false relations. This allows us to determine the causal structure as far as encoded in the probability mass fu…
We propose a novel tensor-based formalism for inferring causal structures from time series. An information theoretical analysis of transfer entropy, shows that transfer entropy results from transmission of information over a set of communication channels. Tensors are the mathematical equivalents of these multi-channel causal channels. A multi-channel causal channel is a generalization of a discrete memoryless channel. Investigation of a system comprising three variables shows that in our formalism, bivariate analysis suffices to differentiate between direct and indirect relations. For this to …
Using ideas from Chu and Bode/Fano theories, we characterize the maximum achievable rate over the single-input single-output wireless communication channels under a restriction on the antenna size at the receiver. By employing circuit-theoretic multiport models for radio communication systems, we derive the information-theoretic limits of compact antennas. We first describe an equivalent Chu's antenna circuit under the physical realizability conditions of its reflection coefficient. Such a design allows us to subsequently compute the achievable rate for a given receive antenna size thereby pro…
Let $W$ be a binary-input memoryless symmetric (BMS) channel with Shannon capacity $I(W)$ and fix any $α> 0$. We construct, for any sufficiently small $δ> 0$, binary linear codes of block length $O(1/δ^{2+α})$ and rate $I(W)-δ$ that enable reliable communication on $W$ with quasi-linear time encoding and decoding. Shannon's noisy coding theorem established the \emph{existence} of such codes (without efficient constructions or decoding) with block length $O(1/δ^2)$. This quadratic dependence on the gap $δ$ to capacity is known to be best possible. Our result thus yields a constructive version o…
We consider the problem of communicating the state of a dynamical system via a Shannon Gaussian channel. The receiver, which acts as both a decoder and estimator, observes the noisy measurement of the channel output and makes an optimal estimate of the state of the dynamical system in the minimum mean square sense. The transmitter observes a possibly noisy measurement of the state of the dynamical system. These measurements are then used to encode the message to be transmitted over a noisy Gaussian channel, where a per sample power constraint is imposed on the transmitted message. Thus, we get…
We explore two fundamental questions at the intersection of sampling theory and information theory: how channel capacity is affected by sampling below the channel's Nyquist rate, and what sub-Nyquist sampling strategy should be employed to maximize capacity. In particular, we derive the capacity of sampled analog channels for three prevalent sampling strategies: sampling with filtering, sampling with filter banks, and sampling with modulation and filter banks. These sampling mechanisms subsume most nonuniform sampling techniques applied in practice. Our analyses illuminate interesting connecti…
In this paper, we characterize the information-theoretic capacity scaling of wireless ad hoc networks with $n$ randomly distributed nodes. By using an exact channel model from Maxwell's equations, we successfully resolve the conflict in the literature between the linear capacity scaling by Özgür et al. and the degrees of freedom limit given as the ratio of the network diameter and the wavelength $λ$ by Franceschetti et al. In dense networks where the network area is fixed, the capacity scaling is given as the minimum of $n$ and the degrees of freedom limit $λ^{-1}$ to within an arbitrarily sma…
This paper introduces a novel framework and corresponding methods for sampling and reconstruction of sparse signals in shift-invariant (SI) spaces. We reinterpret the random demodulator, a system that acquires sparse bandlimited signals, as a system for the acquisition of linear combinations of the samples in the SI setting with the box function as the sampling kernel. The sparsity assumption is exploited by the compressive sensing (CS) paradigm for a recovery of the SI samples from a reduced set of measurements. The SI samples are subsequently filtered by a discrete-time correction filter to …
The capacity of finite state channels (FSCs) has been established as the limit of a sequence of multi-letter expressions only and, despite tremendous effort, a corresponding finite-letter characterization remains unknown to date. This paper analyzes the capacity of FSCs from a fundamental, algorithmic point of view by studying whether or not the corresponding achievability and converse bounds on the capacity can be computed algorithmically. For this purpose, the concept of Turing machines is used which provide the fundamental performance limits of digital computers. To this end, computable con…
This paper studies the interference channel with a cognitive relay (ICCR) under delayed feedback. Three types of delayed feedback are studied: delayed channel state information at the transmitter (CSIT), delayed output feedback, and delayed Shannon feedback. Outer bounds are derived for the DoF region of the two-user multiple-input multiple-output (MIMO) ICCR with delayed feedback as well as without feedback. For the single-input single-output (SISO) scenario, optimal schemes are proposed based on retrospective interference alignment. It is shown that while a cognitive relay without feedback c…
In this paper, we define the power region as the set of power allocations for K users such that everybody meets a minimum signal-to-interference ratio (SIR). The SIR is modeled in a multiuser CDMA system with fixed linear receiver and signature sequences. We show that the power region is convex in linear and logarithmic scale. It furthermore has a componentwise minimal element. Power constraints are included by the intersection with the set of all viable power adjustments. In this framework, we aim at minimizing the total expended power by minimizing a componentwise monotone functional. If the…
This paper develops a new distributed power control scheme for a power splitting-based interference channel (IFC) with simultaneous wireless information and power transfer (SWIPT). The considered IFC consists of multiple source-destination pairs. Each destination splits its received signal into two parts for information decoding and energy harvesting (EH), respectively. Each pair adjusts its transmit power and power splitting ratio to meet both the signal-to-interference-plus-noise ratio (SINR) and EH constraints at its corresponding destination. To characterize rational behaviors of source-de…
In part I, we reviewed how Shannon's classical notion of capacity is not sufficient to characterize a noisy communication channel if the channel is intended to be used as part of a feedback loop to stabilize an unstable scalar linear system. While classical capacity is not enough, a sense of capacity (parametrized by reliability) called "anytime capacity" is both necessary and sufficient for channel evaluation in this context. The rate required is the log of the open-loop system gain and the required reliability comes from the desired sense of stability. Sufficiency is maintained even in cases…
Rail transportation, especially, high-speed rails (HSR), is an important infrastructure for the development of national economy and the promotion of passenger experience. Due to the large bandwidth, millimeter wave (mmWave) communication is regarded as a promising technology to meet the demand of high data rates. However, since mmWave communication has the characteristic of high attenuation, mobile relay (MR) is considered in this paper. Also, full-duplex (FD) communications have been proposed to improve the spectral efficiency. However, because of the high speed, as well as the problem of pen…
Prediction-Powered Communication with Distortion Guarantees
King’s Communications, Learning & Information Processing (KCLIP) lab within the Centre for Intelligent Information Processing Systems (CIIPS), Department of Engineering, King’s College London（国王学院通信、学习与信息处理（KCLIP）实验室，位于智能信息处理系统中心（CIIPS）内，工程系，伦敦国王学院）；Department of Electrical Engineering, Chalmers University of Technology（电子工程系，查尔姆斯理工大学）；Department of Electronic Systems, Aalborg University（电子系统系，奥尔堡大学）
