We propose a nonuniform quantized decoder for polar codes. The design metric of the quantizers is to minimize the distortion incurred by quantization. The quantizers are obtained via dynamic programming and the optimality of the quantizer is proved as well. Simulation results show that the error correction performance degradation of the proposed nonuniform quantized decoder is less than 0.1 dB compared to conventional float-point decoders under 5-bit nonuniform quantization, outperforming previous uniform quantized decoders for polar codes significantly.
We analyze a new group testing scheme, termed semi-quantitative group testing, which may be viewed as a concatenation of an adder channel and a discrete quantizer. Our focus is on non-uniform quantizers with arbitrary thresholds. For the most general semi-quantitative group testing model, we define three new families of sequences capturing the constraints on the code design imposed by the choice of the thresholds. The sequences represent extensions and generalizations of Bh and certain types of super-increasing and lexicographically ordered sequences, and they lead to code structures amenable …
In this paper, adaptive estimation based on noisy quantized observations is studied. A low complexity adaptive algorithm using a quantizer with adjustable input gain and offset is presented. Three possible scalar models for the parameter to be estimated are considered: constant, Wiener process and Wiener process with deterministic drift. After showing that the algorithm is asymptotically unbiased for estimating a constant, it is shown, in the three cases, that the asymptotic mean squared error depends on the Fisher information for the quantized measurements. It is also shown that the loss of p…
In this paper, we consider distributed maximum likelihood estimation (MLE) with dependent quantized data under the assumption that the structure of the joint probability density function (pdf) is known, but it contains unknown deterministic parameters. The parameters may include different vector parameters corresponding to marginal pdfs and parameters that describe dependence of observations across sensors. Since MLE with a single quantizer is sensitive to the choice of thresholds due to the uncertainty of pdf, we concentrate on MLE with multiple groups of quantizers (which can be determined b…
Given a channel having binary input X = (x_1, x_2) having the probability distribution p_X = (p_{x_1}, p_{x_2}) that is corrupted by a continuous noise to produce a continuous output y \in Y = R. For a given conditional distribution p(y|x_1) = ϕ_1(y) and p(y|x_2) = ϕ_2(y), one wants to quantize the continuous output y back to the final discrete output Z = (z_1, z_2, ..., z_N) with N \leq 2 such that the mutual information between input and quantized-output I(X; Z) is maximized while the probability of the quantized-output p_Z = (p_{z_1}, p_{z_2}, ..., p_{z_N}) has to satisfy a certain constrai…
In this paper, we investigate the quantization of the output of a binary input discrete memoryless channel that maximizing the mutual information between the input and the quantized output under an entropy-constrained of the quantized output. A polynomial time algorithm is introduced that can find the truly global optimal quantizer. These results hold for binary input channels with an arbitrary number of quantized output. Finally, we extend these results to binary input continuous output channels and show a sufficient condition such that a single threshold quantizer is an optimal quantizer. Bo…
The paper studies the problem of distributed average consensus in sensor networks with quantized data and random link failures. To achieve consensus, dither (small noise) is added to the sensor states before quantization. When the quantizer range is unbounded (countable number of quantizer levels), stochastic approximation shows that consensus is asymptotically achieved with probability one and in mean square to a finite random variable. We show that the meansquared error (m.s.e.) can be made arbitrarily small by tuning the link weight sequence, at a cost of the convergence rate of the algorit…
Channel output quantization plays a vital role in high-speed emerging memories such as the spin-torque transfer magnetic random access memory (STT-MRAM), where high-precision analog-to-digital converters (ADCs) are not applicable. In this paper, we investigate the design of the 1-bit quantizer which is highly suitable for practical applications. We first propose a quantized channel model for STT-MRAM. We then analyze various information theoretic bounds for the quantized channel, including the channel capacity, cutoff rate, and the Polyanskiy-Poor-Verdú (PPV) finite-length performance bound. B…
We study quantized beamforming in wireless amplify-and-forward relay-interference networks with any number of transmitters, relays, and receivers. We design the quantizer of the channel state information to minimize the probability that at least one receiver incorrectly decodes its desired symbol(s). Correspondingly, we introduce a generalized diversity measure that encapsulates the conventional one as the first-order diversity. Additionally, it incorporates the second-order diversity, which is concerned with the transmitter power dependent logarithmic terms that appear in the error rate expre…
The topic of this study is the outage behavior of multiple-antenna slow fading channels with quantized feedback and partial power control. A fixed-rate communication system is considered. It is known from the literature that with error-free feedback, the outage-optimal quantizer for power control has a circular structure. Moreover, the diversity gain of the system increases polynomially with the cardinality of the power control codebook. Here, a similar system is studied, but when the feedback link is error-prone. We prove that in the high-SNR regime, the optimal quantizer structure with noisy…
We consider vector-quantized (VQ) transmission of compressed sensing (CS) measurements over noisy channels. Adopting mean-square error (MSE) criterion to measure the distortion between a sparse vector and its reconstruction, we derive channel-optimized quantization principles for encoding CS measurement vector and reconstructing sparse source vector. The resulting necessary optimal conditions are used to develop an algorithm for training channel-optimized vector quantization (COVQ) of CS measurements by taking the end-to-end distortion measure into account.
Randomized (dithered) quantization is a method capable of achieving white reconstruction error independent of the source. Dithered quantizers have traditionally been considered within their natural setting of uniform quantization. In this paper we extend conventional dithered quantization to nonuniform quantization, via a subterfage: dithering is performed in the companded domain. Closed form necessary conditions for optimality of the compressor and expander mappings are derived for both fixed and variable rate randomized quantization. Numerically, mappings are optimized by iteratively imposin…
In this paper, distributed Bayesian detection problems with unknown prior probabilities of hypotheses are considered. The sensors obtain observations which are conditionally dependent across sensors and their probability density functions (pdf) are not exactly known. The observations are quantized and are sent to the fusion center. The fusion center fuses the current quantized observations and makes a final decision. It also designs (updated) quantizers to be used at the sensors and the fusion rule based on all previous quantized observations. Information regarding updated quantizers is sent b…
Massive multiple-input multiple-output (MIMO) systems, which utilize a large number of antennas at the base station, are expected to enhance network throughput by enabling improved multiuser MIMO techniques. To deploy many antennas in reasonable form factors, base stations are expected to employ antenna arrays in both horizontal and vertical dimensions, which is known as full-dimension (FD) MIMO. The most popular two-dimensional array is the uniform planar array (UPA), where antennas are placed in a grid pattern. To exploit the full benefit of massive MIMO in frequency division duplexing (FDD)…
This paper studies the problem of reconstructing sparse or compressible signals from compressed sensing measurements that have undergone nonuniform quantization. Previous approaches to this Quantized Compressed Sensing (QCS) problem based on Gaussian models (bounded l2-norm) for the quantization distortion yield results that, while often acceptable, may not be fully consistent: re-measurement and quantization of the reconstructed signal do not necessarily match the initial observations. Quantization distortion instead more closely resembles heteroscedastic uniform noise, with variance dependin…
The goal of combining beamforming and space-time coding in this work is to obtain full-diversity order and to provide additional received power (array gain) compared to conventional space-time codes. In our system, we consider a quasi-static fading environment and we incorporate both high-rate and low-rate feedback channels with possible feedback errors. To utilize feedback information, a class of code constellations is proposed, inspired from orthogonal designs and precoded space-time block codes, which is called generalized partly orthogonal designs or generalized PODs. Furthermore, to model…
We consider the problem of distributed estimation under the Bayesian criterion and explore the design of optimal quantizers in such a system. We show that, for a conditionally unbiased and efficient estimator at the fusion center and when local observations have identical distributions, it is optimal to partition the local sensors into groups, with all sensors within a group using the same quantization rule. When all the sensors use identical number of decision regions, use of identical quantizers at the sensors is optimal. When the network is constrained by the capacity of the wireless multip…
We identify a practical vector quantizer design problem where any fixed-length quantizer (FLQ) yields non-zero distortion at any finite rate, while there is a variable-length quantizer (VLQ) that can achieve zero distortion with arbitrarily low rate. The problem arises in a $t \times 1$ multiple-antenna fading channel where we would like to minimize the channel outage probability by employing beamforming via quantized channel state information at the transmitter (CSIT). It is well-known that in such a scenario, finite-rate FLQs cannot achieve the full-CSIT (zero distortion) outage performance.…
The capacity of a Multiple-Input Multiple-Output (MIMO) channel in which the antenna outputs are processed by an analog linear combining network and quantized by a set of threshold quantizers is studied. The linear combining weights and quantization thresholds are selected from a set of possible configurations as a function of the channel matrix. The possible configurations of the combining network model specific analog receiver architectures, such as single antenna selection, sign quantization of the antenna outputs or linear processing of the outputs. An interesting connection between the ca…
Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal -- sometimes greatly so. This paper develops generalized approximate message passing (GAMP) algorithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowing the linear expansion to be overcomplete or undercomplete and the scalar quantization to be regular or non-regular. GAMP is a recently-developed class of algorithms that uses Gaussian approximations in belief propagation and allows arbitrary separable inpu…
We consider a channel with discrete binary input X that is corrupted by a given continuous noise to produce a continuous-valued output Y. A quantizer is then used to quantize the continuous-valued output Y to the final binary output Z. The goal is to design an optimal quantizer Q* and also find the optimal input distribution p*(X) that maximizes the mutual information I(X; Z) between the binary input and the binary quantized output. A linear time complexity searching procedure is proposed. Based on the properties of the optimal quantizer and the optimal input distribution, we reduced the searc…
Communication of quantized information is frequently followed by a computation. We consider situations of \emph{distributed functional scalar quantization}: distributed scalar quantization of (possibly correlated) sources followed by centralized computation of a function. Under smoothness conditions on the sources and function, companding scalar quantizer designs are developed to minimize mean-squared error (MSE) of the computed function as the quantizer resolution is allowed to grow. Striking improvements over quantizers designed without consideration of the function are possible and are larg…
Compressed sensing (CS) is on recovery of high dimensional signals from their low dimensional linear measurements under a sparsity prior and digital quantization of the measurement data is inevitable in practical implementation of CS algorithms. In the existing literature, the quantization error is modeled typically as additive noise and the multi-bit and 1-bit quantized CS problems are dealt with separately using different treatments and procedures. In this paper, a novel variational Bayesian inference based CS algorithm is presented, which unifies the multi- and 1-bit CS processing and is ap…
Model-Driven Deep Learning for Distributed Detection with Low-Bit Quantization
The Hong Kong University of Science and Technology（香港科技大学）；Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳先进研究院）；School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）
Millimeter wave systems suffer from high power consumption and are constrained to use low resolution quantizers --digital to analog and analog to digital converters (DACs and ADCs). However, low resolution quantization leads to reduced data rate and increased out-of-band emission noise. In this paper, a multiple-input multiple-output (MIMO) system with linear transceivers using low resolution DACs and ADCs is considered. An information-theoretic analysis of the system to model the effect of quantization on spectrospatial power distribution and capacity of the system is provided. More precisely…
Quantization plays a critical role in digital signal processing systems. Quantizers are typically designed to obtain an accurate digital representation of the input signal, operating independently of the system task, and are commonly implemented using serial scalar analog-to-digital converters (ADCs). In this work, we study hardware-limited task-based quantization, where a system utilizing a serial scalar ADC is designed to provide a suitable representation in order to allow the recovery of a parameter vector underlying the input signal. We propose hardware-limited task-based quantization syst…
We address the issue of applying quantized compressed sensing (CS) on low-energy telemonitoring. So far, few works studied this problem in applications where signals were only approximately sparse. We propose a two-stage data compressor based on quantized CS, where signals are compressed by compressed sensing and then the compressed measurements are quantized with only 2 bits per measurement. This compressor can greatly reduce the transmission bit-budget. To recover signals from underdetermined, quantized measurements, we develop a Bayesian De-quantization algorithm. It can exploit both the mo…
Estimation of a location parameter based on noisy and binary quantized measurements is considered in this letter. We study the behavior of the Cramer-Rao bound as a function of the quantizer threshold for different symmetric unimodal noise distributions. We show that, in some cases, the intuitive choice of threshold position given by the symmetry of the problem, placing the threshold on the true parameter value, can lead to locally worst estimation performance.
Kronos Research（克罗诺斯研究公司）；California Institute of Technology（加州理工学院）
Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay constraints. The server receives all its information about the problem instance from the worker via a rate-limited noiseless communication channel. We introduce the principle we call Differential Quantization (DQ) that prescribes compensating the past quantization errors to direct the descent trajectory of a quantized algorithm towards that of its unquantized counterpa…
The error floor phenomenon observed with LDPC codes and their graph-based, iterative, message-passing (MP) decoders is commonly attributed to the existence of error-prone substructures -- variously referred to as near codewords, trapping sets, absorbing sets, or pseudocodewords -- in a Tanner graph representation of the code. Many approaches have been proposed to lower the error floor by designing new LDPC codes with fewer such substructures or by modifying the decoding algorithm. Using a theoretical analysis of iterative MP decoding in an idealized trapping set scenario, we show that a contri…
In this work, we investigate guessing random additive noise decoding (GRAND) with quantized soft input. First, we analyze the achievable rate of ordered reliability bits GRAND (ORBGRAND), which uses the rank order of the reliability as quantized soft information. We show that multi-line ORBGRAND can approach capacity for any signal-to-noise ratio (SNR). We then introduce discretized soft GRAND (DSGRAND), which uses information from a conventional quantizer. Simulation results show that DSGRAND well approximates maximum-likelihood (ML) decoding with a number of quantization bits that is in line…
There have been a number of studies on sparse signal recovery from one-bit quantized measurements. Nevertheless, little attention has been paid to the choice of the quantization thresholds and its impact on the signal recovery performance. This paper examines the problem of one-bit quantizer design for sparse signal recovery. Our analysis shows that the magnitude ambiguity that ever plagues conventional one-bit compressed sensing methods can be resolved, and an arbitrarily small reconstruction error can be achieved by setting the quantization thresholds close enough to the original data sample…
Compressed sensing (CS) is a signal acquisition paradigm to simultaneously acquire and reduce dimension of signals that admit sparse representations. When such a signal is acquired according to the principles of CS, the measurements still take on values in the continuum. In today's "digital" world, a subsequent quantization step, where these measurements are replaced with elements from a finite set is crucial. We focus on one of the approaches that yield efficient quantizers for CS: $ΣΔ$ quantization, followed by a one-stage tractable reconstruction method, which was developed by Saab et al. w…
The University of Hong Kong（香港大学）；CentraleSupélec（中央理工-高等电力学院）；University of Paris-Saclay（巴黎萨克雷大学）
Edge machine learning involves the deployment of learning algorithms at the wireless network edge so as to leverage massive mobile data for enabling intelligent applications. The mainstream edge learning approach, federated learning, has been developed based on distributed gradient descent. Based on the approach, stochastic gradients are computed at edge devices and then transmitted to an edge server for updating a global AI model. Since each stochastic gradient is typically high-dimensional (with millions to billions of coefficients), communication overhead becomes a bottleneck for edge learn…
The polar receiver architecture is a receiver design that captures the envelope and phase information of the signal rather than its in-phase and quadrature components. Several studies have demonstrated the robustness of polar receivers to phase noise and other nonlinearities. Yet, the information-theoretic limits of polar receivers with finite-precision quantizers have not been investigated in the literature. The main contribution of this work is to identify the optimal signaling strategy for the additive white Gaussian noise (AWGN) channel with polar quantization at the output. More precisely…
A popular approach to learning encoders for lossy compression is to use additive uniform noise during training as a differentiable approximation to test-time quantization. We demonstrate that a uniform noise channel can also be implemented at test time using universal quantization (Ziv, 1985). This allows us to eliminate the mismatch between training and test phases while maintaining a completely differentiable loss function. Implementing the uniform noise channel is a special case of the more general problem of communicating a sample, which we prove is computationally hard if we do not make a…
This paper investigates the sum spectral efficiency maximization problem in downlink multiuser multiple-input multiple-output (MIMO) systems with low-resolution quantizers at an access point (AP) and users. In particular, we consider rate-splitting multiple access (RSMA) to enhance spectral efficiency by offering opportunities to boost achievable degrees of freedom. Optimizing RSMA precoders, however, is highly challenging due to the minimum rate constraint when determining the rate of the common stream. The quantization errors coupled with the precoders further make the problem more complicat…
In this paper, we propose a finite-precision decoding method that features the three steps of Reconstruction, Computation, and Quantization (RCQ). Unlike Mutual-Information-Maximization Quantized Belief Propagation (MIM-QBP), RCQ can approximate either belief propagation or Min-Sum decoding. One problem faced by MIM-QBP decoder is that it cannot work well when the fraction of degree-2 variable nodes is large. However, sometimes a large fraction of degree-2 variable nodes is necessary for a fast encoding structure, as seen in the IEEE 802.11 standard and the DVB-S2 standard. In contrast, the pr…
Quantizers play a critical role in digital signal processing systems. Recent works have shown that the performance of quantization systems acquiring multiple analog signals using scalar analog-to-digital converters (ADCs) can be significantly improved by properly processing the analog signals prior to quantization. However, the design of such hybrid quantizers is quite complex, and their implementation requires complete knowledge of the statistical model of the analog signal, which may not be available in practice. In this work we design data-driven task-oriented quantization systems with scal…
Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization
Fraunhofer Institute for Telecommunications – Heinrich Hertz Institute（弗劳恩霍夫电信研究所——海因里希·赫兹研究所）；Institute of Computer Science（计算机科学研究所）；Free University of Berlin（柏林自由大学）；Department of Telecommunication Systems（电信系统系）
