A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein--Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. The approach allows for the derivation of a version of the Cramér--Rao bound that is specific to a given Bregman divergence. The new generalization of the Cramér--Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. The effectiveness of the new bound is evaluated in the Poisson noise setting and the Binomial noise setting.
A lower bound is an important tool for predicting the performance that an estimator can achieve under a particular statistical model. Bayesian bounds are a kind of such bounds which not only utilizes the observation statistics but also includes the prior model information. In reality, however, the true model generating the data is either unknown or simplified when deriving estimators, which motivates the works to derive estimation bounds under modeling mismatch situations. This paper provides a derivation of a Bayesian Cramér-Rao bound under model misspecification, defining important concepts …
Using differential geometry, I derive a form of the Bayesian Cramér-Rao bound that remains invariant under reparametrization. With the invariant formulation at hand, I find the optimal and naturally invariant bound among the Gill-Levit family of bounds. By assuming that the prior probability density is the square of a wavefunction, I also express the bounds in terms of functionals that are quadratic with respect to the wavefunction and its gradient. The problem of finding an unfavorable prior to tighten the bound for minimax estimation is shown, in a special case, to be equivalent to finding t…
In this paper, we derive Hybrid, Bayesian and Marginalized Cramér-Rao lower bounds (HCRB, BCRB and MCRB) for the single and multiple measurement vector Sparse Bayesian Learning (SBL) problem of estimating compressible vectors and their prior distribution parameters. We assume the unknown vector to be drawn from a compressible Student-t prior distribution. We derive CRBs that encompass the deterministic or random nature of the unknown parameters of the prior distribution and the regression noise variance. We extend the MCRB to the case where the compressible vector is distributed according to a…
The variance and the entropy power of a continuous random variable are bounded from below by the reciprocal of its Fisher information through the Cramér-Rao bound and the Stam's inequality respectively. In this note, we introduce the Fisher information for discrete random variables and derive the discrete Cramér-Rao-type bound and the discrete Stam's inequality.
To any parametric family of states of a finite level quantum system we associate a space of Fisher maps and introduce the natural notions of Cramér-Rao-Bhattacharya tensor and Fisher information form. This leads us to an abstract Cramér-Rao-Bhattacharya lower bound for the covariance matrix of any finite number of unbiased estimators of parameteric functions. A number of illustrative examples is included. Modulo technical assumptions of various kinds our methods can be applied to infinite level quantum systems as well as parametric families of classical probability distributions on Borel space…
The relative $α$-entropy is the Rényi analog of relative entropy and arises prominently in information-theoretic problems. Recent information geometric investigations on this quantity have enabled the generalization of the Cramér-Rao inequality, which provides a lower bound for the variance of an estimator of an escort of the underlying parametric probability distribution. However, this framework remains unexamined in the Bayesian framework. In this paper, we propose a general Riemannian metric based on relative $α$-entropy to obtain a generalized Bayesian Cramér-Rao inequality. This establish…
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior, we obtain a Bayesian Cramér-Rao bound which holds for any (possibly biased) estimator and, unlike the van Trees inequality, does not depend on the Fisher information of the prior.
The goal of this paper is to characterize the best achievable performance for the problem of estimating an unknown parameter having a sparse representation. Specifically, we consider the setting in which a sparsely representable deterministic parameter vector is to be estimated from measurements corrupted by Gaussian noise, and derive a lower bound on the mean-squared error (MSE) achievable in this setting. To this end, an appropriate definition of bias in the sparse setting is developed, and the constrained Cramer-Rao bound (CRB) is obtained. This bound is shown to equal the CRB of an estimat…
In the present work, we show how the generalized Cramér-Rao inequality for the estimation of a parameter, presented in a recent paper, can be extended to the mutidimensional case with general norms on $\mathbb{R}^{n}$, and to a wider context. As a particular case, we obtain a new multidimensional Cramér-Rao inequality which is saturated by generalized $q$-Gaussian distributions. We also give another related Cramér-Rao inequality, for a general norm, which is saturated as well by these distributions. Finally, we derive uncertainty relations from these Cramér-Rao inequalities. These uncertainty …
This paper deals with Cramér-Rao inequalities in the context of nonextensive statistics and in estimation theory. It gives characterizations of generalized q-Gaussian distributions, and introduces generalized versions of Fisher information. The contributions of this paper are (i) the derivation of new extended Cramér-Rao inequalities for the estimation of a parameter, involving general q-moments of the estimation error, (ii) the derivation of Cramér-Rao inequalities saturated by generalized q-Gaussian distributions, (iii) the definition of generalized Fisher informations, (iv) the identificati…
We study the geometry of probability distributions with respect to a generalized family of Csiszár $f$-divergences. A member of this family is the relative $α$-entropy which is also a Rényi analog of relative entropy in information theory and known as logarithmic or projective power divergence in statistics. We apply Eguchi's theory to derive the Fisher information metric and the dual affine connections arising from these generalized divergence functions. This enables us to arrive at a more widely applicable version of the Cramér-Rao inequality, which provides a lower bound for the variance of…
In this paper, we address the theoretical limitations in reconstructing sparse signals (in a known complete basis) using compressed sensing framework. We also divide the CS to non-blind and blind cases. Then, we compute the Bayesian Cramer-Rao bound for estimating the sparse coefficients while the measurement matrix elements are independent zero mean random variables. Simulation results show a large gap between the lower bound and the performance of the practical algorithms when the number of measurements are low.
A Cramer-Rao bound (CRB) for semi-blind channel estimators in redundant block transmission systems is derived. The derived CRB is valid for any system adopting a full-rank linear redundant precoder, including the popular cyclic-prefixed orthogonal frequency-division multiplexing system. Simple forms of CRBs for multiple complex parameters, either unconstrained or constrained by a holomorphic function, are also derived, which facilitate the CRB derivation of the problem of interest. The derived CRB is a lower bound on the variance of any unbiased semi-blind channel estimator, and can serve as a…
The analytic expression of CRLB and the maximum likelihood estimator for the sample frequency correlation matrices in doubly selective fading channels for OFDM systems are reported in this paper. According to the analytical and numerical results, the amount of samples affects the average mean square error dominantly while the SNR and the Doppler spread do negligibly.
In this paper, the Cramer Rao bound (CRB) for range estimation between two underwater nodes is calculated under a Gaussian noise assumption on the measurements. The nodes can measure their depths, their mutual time of flight, and they have access to noisy sound speed samples at different depths. The effect of each measurement on the CRB will be analyzed, and it will be shown that for long distances, the effect of the sound speed measurement noise is dominant, and its impact depends on the positions of the nodes, actual sound speed profile, the number of sound speed samples, and the depths at w…
We propose a modified $χ^β$-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimation error. In the case of a location parameter, we obtain new characterizations of the generalized $q$-Gaussians, for instance as the distribution with a given moment that minimizes the generalized Fisher information. Finally we indicate how the generalized Fisher information can…
In this paper, we derive the Cramer-Rao bound (CRB) for blind channel estimation in redundant block transmission systems, a lower bound for the mean squared error of any blind channel estimators. The derived CRB is valid for any full-rank linear redundant precoder, including both zero-padded (ZP) and cyclic-prefixed (CP) precoders. A simple form of CRBs for multiple complex parameters is also derived and presented which facilitates the CRB derivation of the problem of interest. A comparison is made between the derived CRBs and performances of existing subspace-based blind channel estimators fo…
This paper is mainly dedicated to an extension of the Slepian-Bangs formula to non-circular complex elliptical symmetric (NC-CES) distributions, which is derived from a new stochastic representation theorem. This formula includes the non-circular complex Gaussian and the circular CES (CCES) distributions. Some general relations between the Cramer Rao bound (CRB) under CES and Gaussian distributions are deduced. It is proved in particular that the Gaussian distribution does not always lead to the largest stochastic CRB (SCRB) as many authors tend to believe it. Finally a particular attention is…
The analytic expression of CRLB and the maximum likelihood estimator for spatial correlation matrices in time-varying multipath fading channels for MIMO OFDM systems are reported in this paper. The analytical and numerical results reveal that the amount of samples and the order of frequency selectivity have dominant impact on the CRLB. Moreover, the number of pilot tones, SNR as well as the normalized maximum Doppler spread together influence the effective order of frequency selectivity.
In this paper, we consider the Cramer-Rao lower bound (CRLB) for estimation of a lens-embedded antenna array with deterministic parameters. Unlike CRLB of uniform linear array (ULA), it is noted that CRLB for direction of arrival (DoA) of lens-embedded antenna array is dominated by not only angle but characteristics of lens. Derivation is based on the approximation that amplitude of received signal with lens is approximated to Gaussian function. We confirmed that parameters needed to design a lens can be derived by standard deviation of Gaussian, which represents characteristic of received sig…
In [1], we proved the asymptotic achievability of the Cramér-Rao bound in the compressive sensing setting in the linear sparsity regime. In the proof, we used an erroneous closed-form expression of $ασ^2$ for the genie-aided Cramér-Rao bound $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ from Lemma 3.5, which appears in Eqs. (20) and (29). The proof, however, holds if one avoids replacing $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ by the expression of Lemma 3.5, and hence the claim of the Main Theorem stands true. In Chapter 2 of the Ph. …
To achieve the joint active and passive beamforming gains in the reconfigurable intelligent surface assisted millimeter wave system, the reflected cascade channel needs to be accurately estimated. Many strategies have been proposed in the literature to solve this issue. However, whether the Cramér-Rao lower bound (CRLB) of such estimation is achievable still remains uncertain. To fill this gap, we first convert the channel estimation problem into a sparse signal recovery problem by utilizing the properties of discrete Fourier transform matrix and Kronecker product. Then, a joint typicality bas…
In this paper, we derive for the first time the exact Cramer-Rao bounds (CRBs) on semi-blind channel estimation for amplify-and-forward two-way relay networks. The bounds cover a wide range of modulation schemes that satisfy a certain symmetry condition. In particular, the important classes of PSK and square QAM are covered. For the case square QAM, we also provide simplified expressions that lend themselves more easily to numerical implementation. The derived bounds are used to show that the semi-blind approach, which exploits both the transmitted pilots and the transmitted data symbols, can …
Cramér-Rao Bound Analysis and Near-Optimal Performance of the Synchronous Nyquist-Folding Generalized Eigenvalue Method (SNGEM) for Sub-Nyquist Multi-Tone Parameter Estimation
For frequency division duplex channels, a simple pilot loop-back procedure has been proposed that allows the estimation of the UL & DL channels at an antenna array without relying on any digital signal processing at the terminal side. For this scheme, we derive the maximum likelihood (ML) estimators for the UL & DL channel subspaces, formulate the corresponding Cramér-Rao bounds and show the asymptotic efficiency of both (SVD-based) estimators by means of Monte Carlo simulations. In addition, we illustrate how to compute the underlying (rank-1) SVD with quadratic time complexity by employing t…
We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the non-negative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called Jensen-Bregman distances yields exactly those Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao diverge…
We consider the potential for positioning with a system where antenna arrays are deployed as a large intelligent surface (LIS). We derive Fisher-informations and Cramér-Rao lower bounds (CRLB) in closed-form for terminals along the central perpendicular line (CPL) of the LIS for all three Cartesian dimensions. For terminals at positions other than the CPL, closed-form expressions for the Fisher-informations and CRLBs seem out of reach, and we alternatively provide approximations (in closed-form) which are shown to be very accurate. We also show that under mild conditions, the CRLBs in general …
In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of entropy and Fisher information. More precisely, a generalized Fisher information naturally pops up in the expression of the derivative of the Tsallis entropy. This generalized Fisher information also appears as a special case of a generalized Fisher information for estimation probl…
Cornell University（康奈尔大学）；Sharif University of Technology（谢里夫理工大学）；University of Grenoble（格勒诺布尔大学）；Institut Universitaire de France（法国大学研究院）
Recently, it has been proved in Babadi et al. that in noisy compressed sensing, a joint typical estimator can asymptotically achieve the Cramer-Rao lower bound of the problem.To prove this result, this paper used a lemma,which is provided in Akcakaya et al,that comprises the main building block of the proof. This lemma is based on the assumption of Gaussianity of the measurement matrix and its randomness in the domain of noise. In this correspondence, we generalize the results obtained in Babadi et al by dropping the Gaussianity assumption on the measurement matrix. In fact, by considering the…
Synchronization of rotations is the problem of estimating a set of rotations R_i in SO(n), i = 1, ..., N, based on noisy measurements of relative rotations R_i R_j^T. This fundamental problem has found many recent applications, most importantly in structural biology. We provide a framework to study synchronization as estimation on Riemannian manifolds for arbitrary n under a large family of noise models. The noise models we address encompass zero-mean isotropic noise, and we develop tools for Gaussian-like as well as heavy-tail types of noise in particular. As a main contribution, we derive th…
United States CCDC Army Research Laboratory（美国陆军作战能力发展司令部研究实验室）；Indian Institute of Technology Palakkad（印度理工学院帕拉卡德校区）
We examine the role of information geometry in the context of classical Cramér-Rao (CR) type inequalities. In particular, we focus on Eguchi's theory of obtaining dualistic geometric structures from a divergence function and then applying Amari-Nagoaka's theory to obtain a CR type inequality. The classical deterministic CR inequality is derived from Kullback-Leibler (KL)-divergence. We show that this framework could be generalized to other CR type inequalities through four examples: $α$-version of CR inequality, generalized CR inequality, Bayesian CR inequality, and Bayesian $α$-CR inequality.…
Oscillator phase noise (PHN) and carrier frequency offset (CFO) can adversely impact the performance of orthogonal frequency division multiplexing (OFDM) systems, since they can result in inter carrier interference and rotation of the signal constellation. In this paper, we propose an expectation conditional maximization (ECM) based algorithm for joint estimation of channel, PHN, and CFO in OFDM systems. We present the signal model for the estimation problem and derive the hybrid Cramer-Rao lower bound (HCRB) for the joint estimation problem. Next, we propose an iterative receiver based on an …
We revisit the problem of computing submatrices of the Cramér-Rao bound (CRB), which lower bounds the variance of any unbiased estimator of a vector parameter $\vth$. We explore iterative methods that avoid direct inversion of the Fisher information matrix, which can be computationally expensive when the dimension of $\vth$ is large. The computation of the bound is related to the quadratic matrix program, where there are highly efficient methods for solving it. We present several methods, and show that algorithms in prior work are special instances of existing optimization algorithms. Some of …
We seek to characterize the estimation performance of a sensor network where the individual sensors exhibit the phenomenon of drift, i.e., a gradual change of the bias. Though estimation in the presence of random errors has been extensively studied in the literature, the loss of estimation performance due to systematic errors like drift have rarely been looked into. In this paper, we derive closed-form Fisher Information matrix and subsequently Cramér-Rao bounds (upto reasonable approximation) for the estimation accuracy of drift-corrupted signals. We assume a polynomial time-series as the rep…
The constrained Cramer-Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood (CML) estimator, in the non-asymptotic region these conditions are usually too strict and the commonly-used estimators, such as the CML estimator, do not satisfy them. Therefore, the CCRB may not be a lower bound on the MSE matrix of such estimators. In this paper, we propose a new definition for unbiasedness under constraints, denoted by C-un…
Cramer-Rao Bounds for Activity Detection in Conventional and Fluid Antenna Systems
National Mobile Communications Research Laboratory, Southeast University（东南大学国家移动通信重点实验室）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息工程大学人工智能学院）；Department of Electronic and Electrical Engineering, University College London（伦敦大学学院电子电气工程系）；Yonsei Frontier Lab., Yonsei University（延世大学延世前沿实验室）；School of Integrated Technology, Yonsei University（延世大学综合技术学院）
This paper investigates intelligent reflecting surface (IRS) enabled non-line-of-sight (NLoS) wireless sensing, in which an IRS is deployed to assist an access point (AP) to sense a target in its NLoS region. It is assumed that the AP is equipped with multiple antennas and the IRS is equipped with a uniform linear array. The AP aims to estimate the target's direction-of-arrival (DoA) with respect to the IRS, based on the echo signals from the AP-IRS-target-IRS-AP link. Under this setup, we jointly design the transmit beamforming at the AP and the reflective beamforming at the IRS to minimize t…
In some estimation problems, not all the parameters can be identified, which results in singularity of the Fisher Information Matrix (FIM). The Cramér-Rao Bound (CRB), which is the inverse of the FIM, is then not defined. To regularize the estimation problem, one can impose constraints on the parameters and derive the corresponding CRBs. The correspondence between local identifiability and FIM regularity is studied here. Furthermore the number of FIM singularities is shown to be equal to the number of independent constraints necessary to have a well-defined constrained CRB and local identifiab…
Multiple antenna arrays play a key role in wireless networks for communications but also localization and sensing. The use of large antenna arrays pushes towards a propagation regime in which the wavefront is no longer plane but spherical. This allows to infer the position and orientation of an arbitrary source from the received signal without the need of using multiple anchor nodes. To understand the fundamental limits of large antenna arrays for localization, this paper fusions wave propagation theory with estimation theory, and computes the Cram{é}r-Rao Bound (CRB) for the estimation of the…
Efficient estimation of wideband spectrum is of great importance for applications such as cognitive radio. Recently, sub-Nyquist sampling schemes based on compressed sensing have been proposed to greatly reduce the sampling rate. However, the important issue of quantization has not been fully addressed, particularly for high-resolution spectrum and parameter estimation. In this paper, we aim to recover spectrally-sparse signals and the corresponding parameters, such as frequency and amplitudes, from heavy quantizations of their noisy complex-valued random linear measurements, e.g. only the qua…
