In this paper we propose a new framework for distributed source coding of structured sources, such as sparse signals. Our framework capitalizes on recent advances in the theory of linear inverse problems and signal representations using incoherent projections. Our approach acquires and quantizes incoherent linear measurements of the signal, which are represented as separate bitplanes. Each bitplane is coded using a distributed source code of the appropriate rate, and transmitted. The decoder, starts from the least significant biplane and, using a prediction of the signal as side information, iteratively recovers each bitplane based on the source prediction and the signal, assuming all the previous bitplanes of lower significance have already been recovered. We provide theoretical results guiding the rate selection, relying only on the least squares prediction error of the source. This is in contrast to existing approaches which rely on difficult-to-estimate information-theoretic metrics to set the rate. We validate our approach using simulations on remote-sensing multispectral images, comparing them with existing approaches of similar complexity.
Non-adaptive joint source network coding of correlated sources is discussed in this paper. By studying the information flow in the network, we propose quantized network coding as an alternative for packet forwarding. This technique has both network coding and distributed source coding advantages, simultaneously. Quantized network coding is a combination of random linear network coding in the (infinite) field of real numbers and quantization to cope with the limited capacity of links. With the aid of the results in the literature of compressed sensing, we discuss theoretical and practical feasi…
In this paper, we propose an alternative for routing based packet forwarding, which uses network coding to increase transmission efficiency, in terms of both compression and error resilience. This non-adaptive encoding is called quantized network coding, which involves random linear mapping in the real field, followed by quantization to cope with the finite capacity of the links. At the gateway node, which collects received quantized network coder packets, minimum mean squared error decoding is performed, by using belief propagation in the factor graph representation. Our simulation results sh…
In this paper, mathematical bases for non-adaptive joint source network coding of correlated messages in a Bayesian scenario are studied. Specifically, we introduce one-step Quantized Network Coding (QNC), which is a hybrid combination of network coding and packet forwarding for transmission. Motivated by the work on Bayesian compressed sensing, we derive theoretical guarantees on robust recovery in a one-step QNC scenario. Our mathematical derivations for Gaussian messages express the opportunity of distributed compression by using one-step QNC, as a simplified version of QNC scenario. Our si…
In this paper, we study the data gathering problem in the context of power grids by using a network of sensors, where the sensed data have inter-node redundancy. Specifically, we propose a new transmission method, calledquantized network coding, which performs linear net-work coding in the field of real numbers, and quantization to accommodate the finite capacity of edges. By using the concepts in compressed sensing literature, we propose to use l1-minimization to decode the quantized network coded packets, especially when the number of received packets at the decoder is less than the size of …
In this paper, we study joint network coding and distributed source coding of inter-node dependent messages, with the perspective of compressed sensing. Specifically, the theoretical guarantees for robust $\ell_1$-min recovery of an under-determined set of linear network coded sparse messages are investigated. We discuss the guarantees for $\ell_1$-min decoding of quantized network coded messages, using the proposed local network coding coefficients in \cite{naba}, based on Restricted Isometry Property (RIP) of the resulting measurement matrix. Moreover, the relation between tail probability o…
Universal variable-to-fixed (V-F) length coding of $d$-dimensional exponential family of distributions is considered. We propose an achievable scheme consisting of a dictionary, used to parse the source output stream, making use of the previously-introduced notion of quantized types. The quantized type class of a sequence is based on partitioning the space of minimal sufficient statistics into cuboids. Our proposed dictionary consists of sequences in the boundaries of transition from low to high quantized type class size. We derive the asymptotics of the $ε$-coding rate of our coding scheme fo…
In this paper, we address the design of codes which achieve modulation diversity in block fading single-input single-output (SISO) channels with signal quantization at receiver and low-complexity decoding. With an unquantized receiver, coding based on algebraic rotations is known to achieve modulation coding diversity. On the other hand, with a quantized receiver, algebraic rotations may not guarantee diversity. Through analysis, we propose specific rotations which result in the codewords having equidistant component-wise projections. We show that the proposed coding scheme achieves maximum mo…
This paper provides new error bounds on "consistent" reconstruction methods for signals observed from quantized random projections. Those signal estimation techniques guarantee a perfect matching between the available quantized data and a new observation of the estimated signal under the same sensing model. Focusing on dithered uniform scalar quantization of resolution $δ>0$, we prove first that, given a Gaussian random frame of $\mathbb R^N$ with $M$ vectors, the worst-case $\ell_2$-error of consistent signal reconstruction decays with high probability as $O(\frac{N}{M}\log\frac{M}{\sqrt N})$…
Quantized Indexing is a fast and space-efficient form of enumerative (combinatorial) coding, the strongest among asymptotically optimal universal entropy coding algorithms. The present advance in enumerative coding is similar to that made by arithmetic coding with respect to its unlimited precision predecessor, Elias coding. The arithmetic precision, execution time, table sizes and coding delay are all reduced by a factor O(n) at a redundancy below 2*log(e)/2^g bits/symbol (for n input symbols and g-bit QI precision). Due to its tighter enumeration, QI output redundancy is below that of arithm…
Differential pulse-code modulation (DPCM) is recently coupled with uniform scalar quantization (SQ) to improve the rate-distortion (RD) performance for the block-based quantized compressive sensing (CS) of images. In this framework, for each block's CS measurements, a prediction is generated based on the reconstructed CS measurements of the previous blocks and subtracted from measurements of the current block in the measurement domain. The resulting residual is then quantized by uniform SQ to generate the quantization index. However, the entropy coding is still required to remove the statistic…
The goal of combining beamforming and space-time coding in this work is to obtain full-diversity order and to provide additional received power (array gain) compared to conventional space-time codes. In our system, we consider a quasi-static fading environment and we incorporate both high-rate and low-rate feedback channels with possible feedback errors. To utilize feedback information, a class of code constellations is proposed, inspired from orthogonal designs and precoded space-time block codes, which is called generalized partly orthogonal designs or generalized PODs. Furthermore, to model…
A new coding scheme for image transmission over noisy channel is proposed. Similar to standard image compression, the scheme includes a linear transform followed by successive refinement scalar quantization. Unlike conventional schemes, in the proposed system the quantized transform coefficients are linearly mapped into channel symbols using systematic linear encoders. This fixed-to-fixed length "linear index coding" approach avoids the use of an explicit entropy coding stage (e.g., arithmetic or Huffman coding), which is typically fragile to channel post-decoding residual errors. We use linea…
Integer-forcing source coding has been proposed as a low-complexity method for compression of distributed correlated Gaussian sources. In this scheme, each encoder quantizes its observation using the same fine lattice and reduces the result modulo a coarse lattice. Rather than directly recovering the individual quantized signals, the decoder first recovers a full-rank set of judiciously chosen integer linear combinations of the quantized signals, and then inverts it. It has been observed that the method works very well for "most" but not all source covariance matrices. The present work quantif…
Automating the solutions of multiple network information theory problems, stretching from fundamental concerns such as determining all information inequalities and the limitations of linear codes, to applied ones such as designing coded networks, distributed storage systems, and caching systems, can be posed as polyhedral projections. These problems are demonstrated to exhibit multiple types of polyhedral symmetries. It is shown how these symmetries can be exploited to reduce the complexity of solving these problems through polyhedral projection.
Index codes reduce the number of bits broadcast by a wireless transmitter to a number of receivers with different demands and with side information. It is known that the problem of finding optimal linear index codes is NP-hard. We investigate the performance of different heuristics based on rank minimization and matrix completion methods, such as alternating projections and alternating minimization, for constructing linear index codes over the reals. As a summary of our results, the alternating projections method gives the best results in terms of minimizing the number of broadcast bits and co…
Integer-Forcing (IF) is a new framework, based on compute-and-forward, for decoding multiple integer linear combinations from the output of a Gaussian multiple-input multiple-output channel. This work applies the IF approach to arrive at a new low-complexity scheme, IF source coding, for distributed lossy compression of correlated Gaussian sources under a minimum mean squared error distortion measure. All encoders use the same nested lattice codebook. Each encoder quantizes its observation using the fine lattice as a quantizer and reduces the result modulo the coarse lattice, which plays the r…
In this work, the problem of transmitting an i.i.d Gaussian source over an i.i.d Gaussian wiretap channel with an i.i.d Gaussian side information is considered. The intended receiver is assumed to have a certain minimum SNR and the eavesdropper is assumed to have a strictly lower SNR compared to the intended receiver. The objective is minimizing the distortion of source reconstruction at the intended receiver. In this work, it is shown that unlike the Gaussian wiretap channel without side information, Shannon's source-channel separation coding scheme is not optimum in the sense of achieving th…
The method of random projections has become very popular for large-scale applications in statistical learning, information retrieval, bio-informatics and other applications. Using a well-designed coding scheme for the projected data, which determines the number of bits needed for each projected value and how to allocate these bits, can significantly improve the effectiveness of the algorithm, in storage cost as well as computational speed. In this paper, we study a number of simple coding schemes, focusing on the task of similarity estimation and on an application to training linear classifier…
The effects of quantization and coding on the estimation quality of a Gauss-Markov, namely Ornstein-Uhlenbeck, process are considered. Samples are acquired from the process, quantized, and then encoded for transmission using either infinite incremental redundancy or fixed redundancy coding schemes. A fixed processing time is consumed at the receiver for decoding and sending feedback to the transmitter. Decoded messages are used to construct a minimum mean square error (MMSE) estimate of the process as a function of time. This is shown to be an increasing functional of the age-of-information, d…
Random Linear Network Coding (RLNC) has emerged as a powerful tool for robust high-throughput multicast. Projection analysis - a recently introduced technique - shows that the distributed packetized RLNC protocol achieves (order) optimal and perfectly pipelined information dissemination in many settings. In the original approach to RNLC intermediate nodes code together all available information. This requires intermediate nodes to keep considerable data available for coding. Moreover, it results in a coding complexity that grows linearly with the size of this data. While this has been identifi…
This paper studies second-order coding rates for memoryless channels with a state sequence known non-causally at the encoder. In the case of finite alphabets, an achievability result is obtained using constant-composition random coding, and by using a small fraction of the block to transmit the type of the state sequence. For error probabilities less than 1/2, it is shown that the second-order rate improves on an existing one based on i.i.d. random coding. In the Gaussian case (dirty paper coding) with an almost-sure power constraint, an achievability result is obtained used using random codin…
A general method of source coding over expansion is proposed in this paper, which enables one to reduce the problem of compressing an analog (continuous-valued source) to a set of much simpler problems, compressing discrete sources. Specifically, the focus is on lossy compression of exponential and Laplacian sources, which is subsequently expanded using a finite alphabet prior to being quantized. Due to decomposability property of such sources, the resulting random variables post expansion are independent and discrete. Thus, each of the expanded levels corresponds to an independent discrete so…
In this paper, we revisit the sequential source coding framework to analyze fundamental performance limitations of discrete-time stochastic control systems subject to feedback data-rate constraints in finite-time horizon. The basis of our results is a new characterization of the lower bound on the minimum total-rate achieved by sequential codes subject to a total (across time) distortion constraint and a computational algorithm that allocates optimally the rate-distortion for any fixed finite-time horizon. This characterization facilitates the derivation of analytical, non-asymptotic, and fini…
Quantized compressive sensing (QCS) deals with the problem of representing compressive signal measurements with finite precision representation, i.e., a mandatory process in any practical sensor design. To characterize the signal reconstruction quality in this framework, most of the existing theoretical analyses lie heavily on the quantization of sub-Gaussian random projections (e.g., Gaussian or Bernoulli). We show here that a simple uniform scalar quantizer is compatible with a large class of random sensing matrices known to respect, with high probability, the restricted isometry property (R…
We consider a low-complexity version of the Compute and Forward scheme that involves only scaling, offset (dithering removal) and scalar quantization at the relays. The proposed scheme is suited for the uplink of a distributed antenna system where the antenna elements must be very simple and are connected to a oint processor via orthogonal perfect links of given rate R0. We consider the design of non-binary LDPC codes naturally matched to the proposed scheme. Each antenna element performs individual (decentralized) Belief Propagation decoding of its own quantized signal, and sends a linear com…
In this work, the problem of transmitting an i.i.d Gaussian source over an i.i.d Gaussian wiretap channel with an i.i.d Gaussian side information available at the intended receiver is considered. The intended receiver is assumed to have a certain minimum SNR and the eavesdropper is assumed to have a strictly lower SNR, compared to the intended receiver. The objective is to minimize the distortion of source reconstruction at the intended receiver. In this work, it is shown that the source-channel separation coding scheme is optimum in the sense of achieving minimum distortion. Two hybrid digita…
We show how real-number codes can be used to compress correlated sources and establish a new framework for distributed lossy source coding, in which we quantize compressed sources instead of compressing quantized sources. This change in the order of binning and quantization blocks makes it possible to model correlation between continuous-valued sources more realistically and compensate for the quantization error when the sources are completely correlated. We focus on the asymmetric case, i.e., lossy source coding with side information at the decoder, also known as Wyner-Ziv coding. The encodin…
This paper proposes a method for designing error correction codes by combining a known coding scheme with an autoencoder. Specifically, we integrate an LDPC code with a trained autoencoder to develop an error correction code for intractable nonlinear channels. The LDPC encoder shrinks the input space of the autoencoder, which enables the autoencoder to learn more easily. The proposed error correction code shows promising results for one-bit quantization, a challenging case of a nonlinear channel. Specifically, our design gives a waterfall slope bit error rate even with high order modulation fo…
We develop efficient binary (i.e., 1-bit) and multi-bit coding schemes for estimating the scale parameter of $α$-stable distributions. The work is motivated by the recent work on one scan 1-bit compressed sensing (sparse signal recovery) using $α$-stable random projections, which requires estimating of the scale parameter at bits-level. Our technique can be naturally applied to data stream computations for estimating the $α$-th frequency moment. In fact, the method applies to the general scale family of distributions, not limited to $α$-stable distributions. Due to the heavy-tailed nature of $…
A model, called the linear transform network (LTN), is proposed to analyze the compression and estimation of correlated signals transmitted over directed acyclic graphs (DAGs). An LTN is a DAG network with multiple source and receiver nodes. Source nodes transmit subspace projections of random correlated signals by applying reduced-dimension linear transforms. The subspace projections are linearly processed by multiple relays and routed to intended receivers. Each receiver applies a linear estimator to approximate a subset of the sources with minimum mean squared error (MSE) distortion. The mo…
Using physical layer network coding, compute-and-forward is a promising relaying scheme that effectively exploits the interference between users and thus achieves high rates. In this paper, we consider the problem of finding the optimal integer-valued coefficient vector for a relay in the compute-and-forward scheme to maximize the computation rate at that relay. Although this problem turns out to be a shortest vector problem, which is suspected to be NP-hard, we show that it can be relaxed to a series of equality-constrained quadratic programmings. The solutions of the relaxed problems serve a…
We show how real-number codes can be used to compress correlated sources, and establish a new framework for lossy distributed source coding, in which we quantize compressed sources instead of compressing quantized sources. This change in the order of binning and quantization blocks makes it possible to model correlation between continuous-valued sources more realistically and correct quantization error when the sources are completely correlated. The encoding and decoding procedures are described in detail, for discrete Fourier transform (DFT) codes. Reconstructed signal, in the mean squared er…
The aim of this paper is to propose an alternative method to solve a Fault Tolerant Control problem. The model is a linear system affected by a disturbance term: this represents a large class of technological faulty processes. The goal is to make the system able to tolerate the undesired perturbation, i.e., to remove or at least reduce its negative effects; such a task is performed in three steps: the detection of the fault, its identification and the consequent process recovery. When the disturbance function is known to be \emph{quantized} over a finite number of levels, the detection can be …
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension for the practically relevant class of integer-dimensional singular random variables. The proposed entropy definition contains the entropy of discrete random variables and the differential entropy of continuous random variables as special cases. We show that it transforms in a natural manner under Lipschitz functions, and that it is invariant under unitary tr…
It is of paramount importance to achieve efficient data collection in the Internet of Things (IoT). Due to the inherent structural properties (e.g., sparsity) existing in many signals of interest, compressive sensing (CS) technology has been extensively used for data collection in IoT to improve both accuracy and energy efficiency. Apart from the existing works which leverage CS as a channel coding scheme to deal with data loss during transmission, some recent results have started to employ CS as a source coding strategy. The frequently used projection matrices in these CS-based source coding …
It has been shown recently that the dirty-paper coding is the optimal strategy for maximizing the sum rate of multiple-input multiple-output Gaussian broadcast channels (MIMO BC). Moreover, by the channel duality, the nonconvex MIMO BC sum rate problem can be transformed to the convex dual MIMO multiple-access channel (MIMO MAC) problem with a sum power constraint. In this paper, we design an efficient algorithm based on conjugate gradient projection (CGP) to solve the MIMO BC maximum sum rate problem. Our proposed CGP algorithm solves the dual sum power MAC problem by utilizing the powerful c…
Quantized compressive sensing (QCS) deals with the problem of coding compressive measurements of low-complexity signals with quantized, finite precision representations, i.e., a mandatory process involved in any practical sensing model. While the resolution of this quantization clearly impacts the quality of signal reconstruction, there actually exist incompatible combinations of quantization functions and sensing matrices that proscribe arbitrarily low reconstruction error when the number of measurements increases. This work shows that a large class of random matrix constructions known to res…
Random binning is an efficient, yet complex, coding technique for the symmetric L-description source coding problem. We propose an alternative approach, that uses the quantized samples of a bandlimited source as "descriptions". By the Nyquist condition, the source can be reconstructed if enough samples are received. We examine a coding scheme that combines sampling and noise-shaped quantization for a scenario in which only K < L descriptions or all L descriptions are received. Some of the received K-sets of descriptions correspond to uniform sampling while others to non-uniform sampling. This …
We introduce a universal quantization scheme based on random coding, and we analyze its performance. This scheme consists of a source-independent random codebook (typically_mismatched_ to the source distribution), followed by optimal entropy-coding that is_matched_ to the quantized codeword distribution. A single-letter formula is derived for the rate achieved by this scheme at a given distortion, in the limit of large codebook dimension. The rate reduction due to entropy-coding is quantified, and it is shown that it can be arbitrarily large. In the special case of "almost uniform" codebooks (…
This paper focuses on the estimation of low-complexity signals when they are observed through $M$ uniformly quantized compressive observations. Among such signals, we consider 1-D sparse vectors, low-rank matrices, or compressible signals that are well approximated by one of these two models. In this context, we prove the estimation efficiency of a variant of Basis Pursuit Denoise, called Consistent Basis Pursuit (CoBP), enforcing consistency between the observations and the re-observed estimate, while promoting its low-complexity nature. We show that the reconstruction error of CoBP decays li…
In this paper we consider the problem of transmitting a continuous alphabet discrete-time source over an AWGN channel. The design of good curves for this purpose relies on geometrical properties of spherical codes and projections of $N$-dimensional lattices. We propose a constructive scheme based on a set of curves on the surface of a 2N-dimensional sphere and present comparisons with some previous works.
We introduce a discrete network corresponding to any Gaussian wireless network that is obtained by simply quantizing the received signals and restricting the transmitted signals to a finite precision. Since signals in the discrete network are obtained from those of a Gaussian network, the Gaussian network can be operated on the quantization-based digital interface defined by the discrete network. We prove that this digital interface is near-optimal for Gaussian relay networks and the capacities of the Gaussian and the discrete networks are within a bounded gap of O(M^2) bits, where M is the nu…
