We investigate the optimal performance of dense sensor networks by studying the joint source-channel coding problem. The overall goal of the sensor network is to take measurements from an underlying random process, code and transmit those measurement samples to a collector node in a cooperative multiple access channel with potential feedback, and reconstruct the entire random process at the collector node. We provide lower and upper bounds for the minimum achievable expected distortion when the underlying random process is Gaussian. When the Gaussian random process satisfies some general conditions, we evaluate the lower and upper bounds explicitly, and show that they are of the same order for a wide range of power constraints. Thus, for these random processes, under these power constraints, we express the minimum achievable expected distortion as a function of the power constraint. Further, we show that the achievability scheme that achieves the lower bound on the distortion is a separation-based scheme that is composed of multi-terminal rate-distortion coding and amplify-and-forward channel coding. Therefore, we conclude that separation is order-optimal for the dense Gaussian sensor network scenario under consideration, when the underlying random process satisfies some general conditions.
Han-Kobayashi achievable region is the best known inner bound for a general discrete memoryless interference channel. We show that the capacity region can be strictly larger than the Han-Kobayashi region for some channel realizations, and hence the strict sub-optimality of Han-Kobayashi achievable region.
A multiple-descriptions (MD) coding strategy is proposed and an inner bound to the achievable rate-distortion region is derived. The scheme utilizes linear codes. It is shown in two different MD set-ups that the linear coding scheme achieves a larger rate-distortion region than previously known random coding strategies. Furthermore, it is shown via an example that the best known random coding scheme for the set-up can be improved by including additional randomly generated codebooks.
For a multi-user interference channel with multi-antenna transmitters and single-antenna receivers, by restricting each transmitter to Gaussian input and each receiver to a single-user detector, computing the largest achievable rate region amounts to solving a family of non-convex optimization problems. Recognizing the intrinsic connection between the signal power at the intended receiver and the interference power at the unintended receiver, the original family of non-convex optimization problems is converted into a new family of convex optimization problems. It is shown that, for such interf…
In this work, lossy distributed compression of pairs of correlated sources is considered. Conventionally, Shannon's random coding arguments -- using randomly generated unstructured codebooks whose blocklength is taken to be asymptotically large -- are used to derive achievability results. However, it was recently observed that in various multi-terminal communications scenarios, using random codes with constant finite blocklength may lead to improved achievable regions compared to the conventional approach. In other words, in some network communication scenarios, there is a finite optimal value…
We provide a complete characterization of the achievable distortion region for the problem of sending a bivariate Gaussian source over bandwidth-matched Gaussian broadcast channels, where each receiver is interested in only one component of the source. This setting naturally generalizes the simple single Gaussian source bandwidth-matched broadcast problem for which the uncoded scheme is known to be optimal. We show that a hybrid scheme can achieve the optimum for the bivariate case, but neither an uncoded scheme alone nor a separation-based scheme alone is sufficient. We further show that in t…
We consider the source-channel separation architecture for lossy source coding in communication networks. It is shown that the separation approach is optimal in two general scenarios, and is approximately optimal in a third scenario. The two scenarios for which separation is optimal complement each other: the first is when the memoryless sources at source nodes are arbitrarily correlated, each of which is to be reconstructed at possibly multiple destinations within certain distortions, but the channels in this network are synchronized, orthogonal and memoryless point-to-point channels; the sec…
We consider the problem of multiple descriptions (MD) source coding and propose new coding strategies involving both unstructured and structured coding layers. Previously, the most general achievable rate-distortion (RD) region for the $l$-descriptions problem was the Combinatorial Message Sharing with Binning (CMSB) region. The CMSB scheme utilizes unstructured quantizers and unstructured binning. In the first part of the paper, we show that this strategy can be improved upon using more general unstructured quantizers and a more general unstructured binning method. In the second part, structu…
We study the mismatched successive refinement problem where one uses Gaussian codebooks to compress an arbitrary memoryless source with successive minimum Euclidean distance encoding under the quadratic distortion measure. Specifically, we derive achievable refined asymptotics under both the joint excess-distortion probability (JEP) and the separate excess-distortion probabilities (SEP) criteria. For both second-order and moderate deviations asymptotics, we consider two types of codebooks: the spherical codebook where each codeword is drawn independently and uniformly from the surface of a sph…
This paper presents a new achievable rate-distortion region for the general L channel multiple descriptions problem. A well known general region for this problem is due to Venkataramani, Kramer and Goyal (VKG) [1]. Their encoding scheme is an extension of the El-Gamal-Cover (EC) and Zhang- Berger (ZB) coding schemes to the L channel case and includes a combinatorial number of refinement codebooks, one for each subset of the descriptions. As in ZB, the scheme also allows for a single common codeword to be shared by all descriptions. This paper proposes a novel encoding technique involving Combi…
This paper concerns the transmission of two independent Gaussian sources over a two-user decentralized interference channel, assuming that the transmitters are unaware of the instantaneous CSIs. The availability of the channel state information at receivers (CSIR) is considered in two scenarios of perfect and imperfect CSIR. In the imperfect CSIR case, we consider a more practical assumption of having an MMSE estimation of the channel gain at the receivers. In this case, minimizing the expected achievable distortion associated with each link is considered. Due to the absence of CSI at the tran…
For a multi-user interference channel with multi-antenna transmitters and single-antenna receivers, by restricting each receiver to a single-user detector, computing the largest achievable rate region amounts to solving a family of non-convex optimization problems. Recognizing the intrinsic connection between the signal power at the intended receiver and the interference power at the unintended receiver, the original family of non-convex optimization problems is converted into a new family of convex optimization problems. It is shown that, for such interference channels with each receiver impl…
We recently proposed a new coding scheme for the L-channel multiple descriptions (MD) problem for general sources and distortion measures involving `Combinatorial Message Sharing' (CMS) [7] leading to a new achievable rate-distortion region. Our objective in this paper is to establish that this coding scheme strictly subsumes the most popular region for this problem due to Venkataramani, Kramer and Goyal (VKG) [3]. In particular, we show that for a binary symmetric source under Hamming distortion measure, the CMS scheme provides a strictly larger region for all L>2. The principle of the CMS co…
Optimal zero-delay coding (quantization) of $\mathbb{R}^d$-valued linearly generated Markov sources is studied under quadratic distortion. The structure and existence of deterministic and stationary coding policies that are optimal for the infinite horizon average cost (distortion) problem are established. Prior results studying the optimality of zero-delay codes for Markov sources for infinite horizons either considered finite alphabet sources or, for the $\mathbb{R}^d$-valued case, only showed the existence of deterministic and non-stationary Markov coding policies or those which are randomi…
In this paper, we consider the mismatched rate-distortion problem, in which the encoding is done using a codebook, and the encoder chooses the minimum-distortion codeword according to a mismatched distortion function that differs from the true one. For the case of discrete memoryless sources, we establish achievable rate-distortion bounds using multi-user coding techniques, namely, superposition coding and expurgated parallel coding. We study examples where these attain the matched rate-distortion trade-off but a standard ensemble with independent codewords fails to do so. On the other hand, i…
We are interested in how to best communicate a (usually real valued) source to a number of destinations (sinks) over a network with capacity constraints in a collective fidelity metric over all the sinks, a problem which we call joint network-source coding. Unlike the lossless network coding problem, lossy reconstruction of the source at the sinks is permitted. We make a first attempt to characterize the set of all distortions achievable by a set of sinks in a given network. While the entire region of all achievable distortions remains largely an open problem, we find a large, non-trivial subs…
Examined in this paper, is the Gray and Wyner achievable lossy rate region for a tuple of correlated multivariate Gaussian random variables (RVs) $X_1 : Ω\rightarrow {\mathbb R}^{p_1}$ and $X_2 : Ω\rightarrow {\mathbb R}^{p_2}$ with respect to square-error distortions at the two decoders. It is shown that among all joint distributions induced by a triple of RVs $(X_1,X_2, W)$, such that $W : Ω\rightarrow {\mathbb W} $ is the auxiliary RV taking continuous, countable, or finite values, the Gray and Wyner achievable rate region is characterized by jointly Gaussian RVs $(X_1,X_2, W)$ such that $W…
We consider the joint source-channel coding problem of sending a Gaussian source on a K-user Gaussian broadcast channel with bandwidth mismatch. A new outer bound to the achievable distortion region is derived using the technique of introducing more than one additional auxiliary random variable, which was previously used to derive sum-rate lower bound for the symmetric Gaussian multiple description problem. By combining this outer bound with the achievability result based on source-channel separation, we provide approximate characterizations of the achievable distortion region within constant …
We present a new technique to obtain outer-bounds on the capacity region of networks with ultra low-rate feedback. We establish a connection between the achievable rates in the forward channel and the minimum distortion that can be attained over the feedback channel.
We investigate the optimal performance of dense sensor networks by studying the joint source-channel coding problem. The overall goal of the sensor network is to take measurements from an underlying random process, code and transmit those measurement samples to a collector node in a cooperative multiple access channel with imperfect feedback, and reconstruct the entire random process at the collector node. We provide lower and upper bounds for the minimum achievable expected distortion when the underlying random process is Gaussian. In the case where the random process satisfies some general c…
We investigate the optimal performance of dense sensor networks by studying the joint source-channel coding problem. The overall goal of the sensor network is to take measurements from an underlying random process, code and transmit those measurement samples to a collector node in a cooperative multiple access channel with feedback, and reconstruct the entire random process at the collector node. We provide lower and upper bounds for the minimum achievable expected distortion when the underlying random process is stationary and Gaussian. In the case where the random process is also Markovian, …
The problem of sending two correlated vector Gaussian sources over a bandwidth-matched two-user scalar Gaussian broadcast channel is studied in this work, where each receiver wishes to reconstruct its target source under a covariance distortion constraint. We derive a lower bound on the optimal tradeoff between the transmit power and the achievable reconstruction distortion pair. Our derivation is based on a new bounding technique which involves the introduction of appropriate remote sources. Furthermore, it is shown that this lower bound is achievable by a class of hybrid schemes for the spec…
The problem of three-user multiple-access channel (MAC) with noiseless feedback is investigated. A new coding strategy is presented. The coding scheme builds upon the natural extension of the Cover-Leung (CL) scheme; and uses quasi-linear codes. A new single-letter achievable rate region is derived. The new achievable region strictly contains the CL region. This is shown through an example. In this example, the coding scheme achieves optimality in terms of transmission rates. It is shown that any optimality achieving scheme for this example must have a specific algebraic structure. Particularl…
In this paper, we study the asymptotic performance of Abelian group codes for the lossy source coding problem for arbitrary discrete (finite alphabet) memoryless sources as well as the channel coding problem for arbitrary discrete (finite alphabet) memoryless channels. For the source coding problem, we derive an achievable rate-distortion function that is characterized in a single-letter information-theoretic form using the ensemble of Abelian group codes. When the underlying group is a field, it simplifies to the symmetric rate-distortion function. Similarly, for the channel coding problem, w…
Reconfigurable intelligent surfaces (RISs) have recently emerged as a promising technology that can achieve high spectrum and energy efficiency for future wireless networks by integrating a massive number of low-cost and passive reflecting elements. An RIS can manipulate the properties of an incident wave, such as the frequency, amplitude, and phase, and, then, reflect this manipulated wave to a desired destination, without the need for complex signal processing. In this paper, the asymptotic optimality of achievable rate in a downlink RIS system is analyzed under a practical RIS environment w…
In this work, we consider a distributed source coding problem with a joint distortion criterion depending on the sources and the reconstruction. This includes as a special case the problem of computing a function of the sources to within some distortion and also the classic Slepian-Wolf problem, Berger-Tung problem, Wyner-Ziv problem, Yeung-Berger problem and the Ahlswede-Korner-Wyner problem. While the prevalent trend in information theory has been to prove achievability results using Shannon's random coding arguments, using structured random codes offer rate gains over unstructured random co…
This paper studies the minimum achievable source coding rate as a function of blocklength $n$ and probability $ε$ that the distortion exceeds a given level $d$. Tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength. For stationary memoryless sources with separable distortion, the minimum rate achievable is shown to be closely approximated by $R(d) + \sqrt{\frac{V(d)}{n}} Q^{-1}(ε)$, where $R(d)$ is the rate-distortion function, $V(d)$ is the rate dispersion, a characteristic of the source which measures its stochastic variability, and $Q^{-1}(ε)$ …
We derive the optimal signaling for a multiple input multiple output (MIMO) full-duplex two-way channel under the imperfect transmit front-end chain. We characterize the two-way rates of the channel by using a game-theoretical approach, where we focus on the Pareto boundary of the achievable rate region and Nash equilibia (NE). For a MISO full-duplex two-way channel, we prove that beamforming is an optimal transmission strategy which can achieve any point on the Pareto boundary. Furthermore, we present a closed-form expression for the optimal beamforming weights. In our numerical examples we q…
This paper considers a deterministic physical model of massive multiple-input multiple-output (MIMO) systems with uniform linear antenna arrays. It is known that the maximum spatial degrees of freedom is achieved by spacing antenna elements at half the carrier wavelength. The purpose of this paper is to investigate the impacts of spacing antennas more densely than the critical separation. The achievable rates of MIMO systems are evaluated in the large-system limit, where the lengths of transmit and receive antenna arrays tend to infinity with the antenna separations kept constant. The main res…
A single-letter lower bound on the sum rate of multiple description coding with tree-structured distortion constraints is established by generalizing Ozarow's celebrated converse argument through the introduction of auxiliary random variables that form a Markov tree. For the quadratic vector Gaussian case, this lower bound is shown to be achievable by an extended version of the El Gamal-Cover scheme, yielding a complete sum-rate characterization.
This work concerns minimizing the achievable distortion of a Gaussian source over a two-hop block fading channel under mean square-error criterion. It is assumed that there is not a direct link between transmission ends and the communication is carried out through the use of a relay, employing the Decode and Forward (DF) strategy. Moreover, for both of transmitter-to-relay and relay-to-destination links the channel statistics are merely available at the corresponding receivers, while the channel gains are not available at the affiliated transmitters. It is assumed that a Gaussian source is hie…
In this paper, we consider a class of multiterminal source coding problems, each subject to distortion constraints computed using a specific, entropy-based, distortion measure. We provide the achievable rate distortion region for two cases and, in so doing, we demonstrate a relationship between the lossy multiterminal source coding problems with our specific distortion measure and (1) the canonical Slepian-Wolf lossless distributed source coding network, and (2) the Ahlswede-Körner-Wyner source coding with side information problem in which only one of the sources is recovered losslessly.
Rate-distortion formulation is the information-theoretic approach to the study of signal encoding systems. Since a more general approach to model the nonstationarity exhibited by real-world signals is to use appropriately fitted time varying autoregressive (TVAR) models, we have investigated the rate-distortion function for the class of time varying nonstationary signals. In this study, we present formulations of the rate-distortion function for the Gaussian TVAR processes. The rate-distortion function can serve as an information-theoretic bound on the performance achievable by source encoding…
The "water-filling" solution for the quadratic rate-distortion function of a stationary Gaussian source is given in terms of its power spectrum. This formula naturally lends itself to a frequency domain "test-channel" realization. We provide an alternative time-domain realization for the rate-distortion function, based on linear prediction. This solution has some interesting implications, including the optimality at all distortion levels of pre/post filtered vector-quantized differential pulse code modulation (DPCM), and a duality relationship with decision-feedback equalization (DFE) for inte…
The Han-Kobayashi (HK) scheme achieves the best known achievable rate region for the K user interference channel (IC). Simple HK schemes are HK schemes with Gaussian signaling, no time sharing, and no private-common power splitting. The class of simple HK schemes includes the treating interference as noise (TIN) scheme and schemes that involve various levels of interference decoding and cancellation at each receiver. We derive conditions under which simple HK schemes achieve sum capacity for general K user Gaussian ICs. These results generalize existing sum capacity results for the TIN scheme …
We characterize the rate-distortion function for zero-mean stationary Gaussian sources under the MSE fidelity criterion and subject to the additional constraint that the distortion is uncorrelated to the input. The solution is given by two equations coupled through a single scalar parameter. This has a structure similar to the well known water-filling solution obtained without the uncorrelated distortion restriction. Our results fully characterize the unique statistics of the optimal distortion. We also show that, for all positive distortions, the minimum achievable rate subject to the uncorre…
We study the problem of communicating a distributed correlated memoryless source over a memoryless network, from source nodes to destination nodes, under quadratic distortion constraints. We establish the following two complementary results: (a) for an arbitrary memoryless network, among all distributed memoryless sources of a given correlation, Gaussian sources are least compressible, that is, they admit the smallest set of achievable distortion tuples, and (b) for any memoryless source to be communicated over a memoryless additive-noise network, among all noise processes of a given correlati…
We propose a nonuniform quantized decoder for polar codes. The design metric of the quantizers is to minimize the distortion incurred by quantization. The quantizers are obtained via dynamic programming and the optimality of the quantizer is proved as well. Simulation results show that the error correction performance degradation of the proposed nonuniform quantized decoder is less than 0.1 dB compared to conventional float-point decoders under 5-bit nonuniform quantization, outperforming previous uniform quantized decoders for polar codes significantly.
We consider the problem of estimating channel fading coefficients (modeled as a correlated Gaussian vector) via Downlink (DL) training and Uplink (UL) feedback in wideband FDD massive MIMO systems. Using rate-distortion theory, we derive optimal bounds on the achievable channel state estimation error in terms of the number of training pilots in DL ($β_{tr}$) and feedback dimension in UL ($β_{fb}$), with random, spatially isotropic pilots. It is shown that when the number of training pilots exceeds the channel covariance rank ($r$), the optimal rate-distortion feedback strategy achieves an esti…
The simple scheme of treating interference as noise (TIN) is studied in this paper for the 3 x 2 X channel. A new sum-capacity upper bound is derived. This upper bound is transformed into a generalized degrees-of-freedom (GDoF) upper bound, and is shown to coincide with the achievable GDoF of scheme that combines TDMA and TIN for some conditions on the channel parameters. These conditions specify a noisy interference regime which extends noisy interference regimes available in literature. As a by-product, the sum-capacity of the 3 x 2 X channel is characterized within a constant gap in the giv…
We study the following semi-deterministic setting of the joint source-channel coding problem: a deterministic source sequence (a.k.a. individual sequence) is transmitted via a memoryless channel, using delay-limited encoder and decoder, which are both implementable by periodically-varying finite-state machines, and the decoder is granted with access to side information, which is a noisy version of the source sequence. We first derive a lower bound on the achievable expected distortion in terms of the empirical statistics of the source sequence, the number of states of the encoder, the number o…
Rate-distortion-perception theory generalizes Shannon's rate-distortion theory by introducing a constraint on the perceptual quality of the output. The perception constraint complements the conventional distortion constraint and aims to enforce distribution-level consistencies. In this new theory, the information-theoretic limit is characterized by the rate-distortion-perception function. Although a coding theorem for the rate-distortion-perception function has recently been established, the fundamental nature of the optimal coding schemes remains unclear, especially regarding the role of rand…
A communication setup is considered where a transmitter wishes to convey a message to a receiver and simultaneously estimates the state of that receiver through a common waveform. The state is estimated at the transmitter by means of generalized feedback, i.e., a strictly causal channel output, and the known waveform. The scenario at hand is motivated by joint radar and communication, which aims to co-design radar sensing and communication over a shared spectrum and hardware. For the case of memoryless single receiver channels with i.i.d. time-varying state sequences, we fully characterize the…
In this paper, we look at the problem of treating interference as noise (TIN) in the Gaussian interfering multiple access channel (IMAC). The considered network comprises $K$ mutually interfering multiple access channels (MACs), each consisting of two transmitters communicating independent messages to one receiver. We define the TIN scheme for this channel as one in which each MAC performs a power controlled version of its capacity-achieving strategy while treating interference from all other MACs as noise. We characterize an achievable generalized degrees-of-freedom (GDoF) region under the TI…
Characterizing the global maximum of weighted sum-rate (WSR) for the K-user Gaussian interference channel (GIC), with the interference treated as Gaussian noise, is a key problem in wireless communication. However, due to the users' mutual interference, this problem is in general non-convex and thus cannot be solved directly by conventional convex optimization techniques. In this paper, by jointly utilizing the monotonic optimization and rate profile techniques, we develop a new framework to obtain the globally optimal power control and/or beamforming solutions to the WSR maximization problems…
This paper investigates the joint source-channel coding problem of sending two correlated memoryless sources with common part over a memoryless multiple access channel (MAC). An inner bound and two outer bounds on the achievable distortion region are derived. In particular, they respectively recover the existing bounds for several special cases, such as communication without common part, lossless communication, and noiseless communication. When specialized to quadratic Gaussian communication case, transmitting Gaussian sources with Gaussian common part over Gaussian MAC, the inner bound and ou…
Motivated by linear network coding, communication channels perform linear operation over finite fields, namely linear operator channels (LOCs), are studied in this paper. For such a channel, its output vector is a linear transform of its input vector, and the transformation matrix is randomly and independently generated. The transformation matrix is assumed to remain constant for every T input vectors and to be unknown to both the transmitter and the receiver. There are NO constraints on the distribution of the transformation matrix and the field size. Specifically, the optimality of subspace …
We analyze two-user single-antenna fading interference channels with perfect receive channel state information (CSI) and no transmit CSI. For the case of very strong interference, we prove that decoding interference while treating the intended signal as noise, subtracting the result out, and then decoding the desired signal, a process known as "stripping", achieves the diversity-multiplexing tradeoff (DMT) outer bound derived in Akuiyibo and Leveque, Int. Zurich Seminar on Commun., 2008. The proof is constructive in the sense that it provides corresponding code design criteria for DMT optimali…
We consider the information-theoretic optimality of treating inter-cell interference as noise in downlink cellular networks modeled as Gaussian interfering broadcast channels. Establishing a new uplink-downlink duality, we cast the problem in Gaussian interfering broadcast channels to that in Gaussian interfering multiple access channels, and characterize an achievable GDoF region under power control and treating inter-cell interference as (Gaussian) noise. We then identify conditions under which this achievable GDoF region is optimal.
A secrecy system with side information at the decoders is studied in the context of lossy source compression over a noiseless broadcast channel. The decoders have access to different side information sequences that are correlated with the source. The fidelity of the communication to the legitimate receiver is measured by a distortion metric, as is traditionally done in the Wyner-Ziv problem. The secrecy performance of the system is also evaluated under a distortion metric. An achievable rate-distortion region is derived for the general case of arbitrarily correlated side information. Exact bou…
This paper proves the separation between source-network coding and channel coding in networks of noisy, discrete, memoryless channels. We show that the set of achievable distortion matrices in delivering a family of dependent sources across such a network equals the set of achievable distortion matrices for delivering the same sources across a distinct network which is built by replacing each channel by a noiseless, point-to-point bit-pipe of the corresponding capacity. Thus a code that applies source-network coding across links that are made almost lossless through the application of independ…
The rate-distortion-perception function (RDPF; Blau and Michaeli, 2019) has emerged as a useful tool for thinking about realism and distortion of reconstructions in lossy compression. Unlike the rate-distortion function, however, it is unknown whether encoders and decoders exist that achieve the rate suggested by the RDPF. Building on results by Li and El Gamal (2018), we show that the RDPF can indeed be achieved using stochastic, variable-length codes. For this class of codes, we also prove that the RDPF lower-bounds the achievable rate
