The relative $α$-entropy is the Rényi analog of relative entropy and arises prominently in information-theoretic problems. Recent information geometric investigations on this quantity have enabled the generalization of the Cramér-Rao inequality, which provides a lower bound for the variance of an estimator of an escort of the underlying parametric probability distribution. However, this framework remains unexamined in the Bayesian framework. In this paper, we propose a general Riemannian metric based on relative $α$-entropy to obtain a generalized Bayesian Cramér-Rao inequality. This establishes a lower bound for the variance of an unbiased estimator for the $α$-escort distribution starting from an unbiased estimator for the underlying distribution. We show that in the limiting case when the entropy order approaches unity, this framework reduces to the conventional Bayesian Cramér-Rao inequality. Further, in the absence of priors, the same framework yields the deterministic Cramér-Rao inequality.
United States CCDC Army Research Laboratory（美国陆军作战能力发展司令部研究实验室）；Indian Institute of Technology Palakkad（印度理工学院帕拉卡德校区）
We examine the role of information geometry in the context of classical Cramér-Rao (CR) type inequalities. In particular, we focus on Eguchi's theory of obtaining dualistic geometric structures from a divergence function and then applying Amari-Nagoaka's theory to obtain a CR type inequality. The classical deterministic CR inequality is derived from Kullback-Leibler (KL)-divergence. We show that this framework could be generalized to other CR type inequalities through four examples: $α$-version of CR inequality, generalized CR inequality, Bayesian CR inequality, and Bayesian $α$-CR inequality.…
The Cramer-Rao lower bound for the estimation of the affine transformation parameters in a multivariate heteroscedastic errors-in-variables model is derived. The model is suitable for feature-based image registration in which both sets of control points are localized with errors whose covariance matrices vary from point to point. With focus given to the registration of fluorescence microscopy images, the Cramer-Rao lower bound for the estimation of a feature's position (e.g. of a single molecule) in a registered image is also derived. In the particular case where all covariance matrices for th…
We develop a maximum-likelihood based method for regression in a setting where the dependent variable is a random graph and covariates are available on a graph-level. The model generalizes the well-known $β$-model for random graphs by replacing the constant model parameters with regression functions. Cramér-Rao bounds are derived for the undirected $β$-model, the directed $β$-model, and the generalized $β$-model. The corresponding maximum likelihood estimators are compared to the bounds by means of simulations. Moreover, examples are given on how to use the presented maximum likelihood estimat…
A maximum likelihood estimator for fusing the measurements in an inertial sensor array is presented. The maximum likelihood estimator is concentrated and an iterative solution method is presented for the resulting low-dimensional optimization problem. The Cramér-Rao bound for the corresponding measurement fusion problem is derived and used to assess the performance of the proposed method, as well as to analyze how the geometry of the array and sensor errors affect the accuracy of the measurement fusion. The angular velocity information gained from the accelerometers in the array is shown to be…
The variance and the entropy power of a continuous random variable are bounded from below by the reciprocal of its Fisher information through the Cramér-Rao bound and the Stam's inequality respectively. In this note, we introduce the Fisher information for discrete random variables and derive the discrete Cramér-Rao-type bound and the discrete Stam's inequality.
We study the geometry of probability distributions with respect to a generalized family of Csiszár $f$-divergences. A member of this family is the relative $α$-entropy which is also a Rényi analog of relative entropy in information theory and known as logarithmic or projective power divergence in statistics. We apply Eguchi's theory to derive the Fisher information metric and the dual affine connections arising from these generalized divergence functions. This enables us to arrive at a more widely applicable version of the Cramér-Rao inequality, which provides a lower bound for the variance of…
To any parametric family of states of a finite level quantum system we associate a space of Fisher maps and introduce the natural notions of Cramér-Rao-Bhattacharya tensor and Fisher information form. This leads us to an abstract Cramér-Rao-Bhattacharya lower bound for the covariance matrix of any finite number of unbiased estimators of parameteric functions. A number of illustrative examples is included. Modulo technical assumptions of various kinds our methods can be applied to infinite level quantum systems as well as parametric families of classical probability distributions on Borel space…
A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein--Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. The approach allows for the derivation of a version of the Cramér--Rao bound that is specific to a given Bregman divergence. The new generalization of the Cramér--Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. The effectiveness of the new bound is ev…
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior, we obtain a Bayesian Cramér-Rao bound which holds for any (possibly biased) estimator and, unlike the van Trees inequality, does not depend on the Fisher information of the prior.
We propose a modified $χ^β$-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimation error. In the case of a location parameter, we obtain new characterizations of the generalized $q$-Gaussians, for instance as the distribution with a given moment that minimizes the generalized Fisher information. Finally we indicate how the generalized Fisher information can…
In the present work, we show how the generalized Cramér-Rao inequality for the estimation of a parameter, presented in a recent paper, can be extended to the mutidimensional case with general norms on $\mathbb{R}^{n}$, and to a wider context. As a particular case, we obtain a new multidimensional Cramér-Rao inequality which is saturated by generalized $q$-Gaussian distributions. We also give another related Cramér-Rao inequality, for a general norm, which is saturated as well by these distributions. Finally, we derive uncertainty relations from these Cramér-Rao inequalities. These uncertainty …
This paper deals with Cramér-Rao inequalities in the context of nonextensive statistics and in estimation theory. It gives characterizations of generalized q-Gaussian distributions, and introduces generalized versions of Fisher information. The contributions of this paper are (i) the derivation of new extended Cramér-Rao inequalities for the estimation of a parameter, involving general q-moments of the estimation error, (ii) the derivation of Cramér-Rao inequalities saturated by generalized q-Gaussian distributions, (iii) the definition of generalized Fisher informations, (iv) the identificati…
This paper is mainly dedicated to an extension of the Slepian-Bangs formula to non-circular complex elliptical symmetric (NC-CES) distributions, which is derived from a new stochastic representation theorem. This formula includes the non-circular complex Gaussian and the circular CES (CCES) distributions. Some general relations between the Cramer Rao bound (CRB) under CES and Gaussian distributions are deduced. It is proved in particular that the Gaussian distribution does not always lead to the largest stochastic CRB (SCRB) as many authors tend to believe it. Finally a particular attention is…
In this paper, we address the theoretical limitations in reconstructing sparse signals (in a known complete basis) using compressed sensing framework. We also divide the CS to non-blind and blind cases. Then, we compute the Bayesian Cramer-Rao bound for estimating the sparse coefficients while the measurement matrix elements are independent zero mean random variables. Simulation results show a large gap between the lower bound and the performance of the practical algorithms when the number of measurements are low.
The goal of this paper is to characterize the best achievable performance for the problem of estimating an unknown parameter having a sparse representation. Specifically, we consider the setting in which a sparsely representable deterministic parameter vector is to be estimated from measurements corrupted by Gaussian noise, and derive a lower bound on the mean-squared error (MSE) achievable in this setting. To this end, an appropriate definition of bias in the sparse setting is developed, and the constrained Cramer-Rao bound (CRB) is obtained. This bound is shown to equal the CRB of an estimat…
A Cramer-Rao bound (CRB) for semi-blind channel estimators in redundant block transmission systems is derived. The derived CRB is valid for any system adopting a full-rank linear redundant precoder, including the popular cyclic-prefixed orthogonal frequency-division multiplexing system. Simple forms of CRBs for multiple complex parameters, either unconstrained or constrained by a holomorphic function, are also derived, which facilitate the CRB derivation of the problem of interest. The derived CRB is a lower bound on the variance of any unbiased semi-blind channel estimator, and can serve as a…
The analytic expression of CRLB and the maximum likelihood estimator for the sample frequency correlation matrices in doubly selective fading channels for OFDM systems are reported in this paper. According to the analytical and numerical results, the amount of samples affects the average mean square error dominantly while the SNR and the Doppler spread do negligibly.
In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of entropy and Fisher information. More precisely, a generalized Fisher information naturally pops up in the expression of the derivative of the Tsallis entropy. This generalized Fisher information also appears as a special case of a generalized Fisher information for estimation probl…
In this paper, we derive the Cramer-Rao bound (CRB) for blind channel estimation in redundant block transmission systems, a lower bound for the mean squared error of any blind channel estimators. The derived CRB is valid for any full-rank linear redundant precoder, including both zero-padded (ZP) and cyclic-prefixed (CP) precoders. A simple form of CRBs for multiple complex parameters is also derived and presented which facilitates the CRB derivation of the problem of interest. A comparison is made between the derived CRBs and performances of existing subspace-based blind channel estimators fo…
In [1], we proved the asymptotic achievability of the Cramér-Rao bound in the compressive sensing setting in the linear sparsity regime. In the proof, we used an erroneous closed-form expression of $ασ^2$ for the genie-aided Cramér-Rao bound $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ from Lemma 3.5, which appears in Eqs. (20) and (29). The proof, however, holds if one avoids replacing $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ by the expression of Lemma 3.5, and hence the claim of the Main Theorem stands true. In Chapter 2 of the Ph. …
To achieve the joint active and passive beamforming gains in the reconfigurable intelligent surface assisted millimeter wave system, the reflected cascade channel needs to be accurately estimated. Many strategies have been proposed in the literature to solve this issue. However, whether the Cramér-Rao lower bound (CRLB) of such estimation is achievable still remains uncertain. To fill this gap, we first convert the channel estimation problem into a sparse signal recovery problem by utilizing the properties of discrete Fourier transform matrix and Kronecker product. Then, a joint typicality bas…
Cramér-Rao Bound Analysis and Near-Optimal Performance of the Synchronous Nyquist-Folding Generalized Eigenvalue Method (SNGEM) for Sub-Nyquist Multi-Tone Parameter Estimation
For frequency division duplex channels, a simple pilot loop-back procedure has been proposed that allows the estimation of the UL & DL channels at an antenna array without relying on any digital signal processing at the terminal side. For this scheme, we derive the maximum likelihood (ML) estimators for the UL & DL channel subspaces, formulate the corresponding Cramér-Rao bounds and show the asymptotic efficiency of both (SVD-based) estimators by means of Monte Carlo simulations. In addition, we illustrate how to compute the underlying (rank-1) SVD with quadratic time complexity by employing t…
This paper investigates intelligent reflecting surface (IRS) enabled non-line-of-sight (NLoS) wireless sensing, in which an IRS is deployed to assist an access point (AP) to sense a target in its NLoS region. It is assumed that the AP is equipped with multiple antennas and the IRS is equipped with a uniform linear array. The AP aims to estimate the target's direction-of-arrival (DoA) with respect to the IRS, based on the echo signals from the AP-IRS-target-IRS-AP link. Under this setup, we jointly design the transmit beamforming at the AP and the reflective beamforming at the IRS to minimize t…
In some estimation problems, not all the parameters can be identified, which results in singularity of the Fisher Information Matrix (FIM). The Cramér-Rao Bound (CRB), which is the inverse of the FIM, is then not defined. To regularize the estimation problem, one can impose constraints on the parameters and derive the corresponding CRBs. The correspondence between local identifiability and FIM regularity is studied here. Furthermore the number of FIM singularities is shown to be equal to the number of independent constraints necessary to have a well-defined constrained CRB and local identifiab…
Cornell University（康奈尔大学）；Sharif University of Technology（谢里夫理工大学）；University of Grenoble（格勒诺布尔大学）；Institut Universitaire de France（法国大学研究院）
Recently, it has been proved in Babadi et al. that in noisy compressed sensing, a joint typical estimator can asymptotically achieve the Cramer-Rao lower bound of the problem.To prove this result, this paper used a lemma,which is provided in Akcakaya et al,that comprises the main building block of the proof. This lemma is based on the assumption of Gaussianity of the measurement matrix and its randomness in the domain of noise. In this correspondence, we generalize the results obtained in Babadi et al by dropping the Gaussianity assumption on the measurement matrix. In fact, by considering the…
Multiple antenna arrays play a key role in wireless networks for communications but also localization and sensing. The use of large antenna arrays pushes towards a propagation regime in which the wavefront is no longer plane but spherical. This allows to infer the position and orientation of an arbitrary source from the received signal without the need of using multiple anchor nodes. To understand the fundamental limits of large antenna arrays for localization, this paper fusions wave propagation theory with estimation theory, and computes the Cram{é}r-Rao Bound (CRB) for the estimation of the…
We consider the potential for positioning with a system where antenna arrays are deployed as a large intelligent surface (LIS). We derive Fisher-informations and Cramér-Rao lower bounds (CRLB) in closed-form for terminals along the central perpendicular line (CPL) of the LIS for all three Cartesian dimensions. For terminals at positions other than the CPL, closed-form expressions for the Fisher-informations and CRLBs seem out of reach, and we alternatively provide approximations (in closed-form) which are shown to be very accurate. We also show that under mild conditions, the CRLBs in general …
Synchronization of rotations is the problem of estimating a set of rotations R_i in SO(n), i = 1, ..., N, based on noisy measurements of relative rotations R_i R_j^T. This fundamental problem has found many recent applications, most importantly in structural biology. We provide a framework to study synchronization as estimation on Riemannian manifolds for arbitrary n under a large family of noise models. The noise models we address encompass zero-mean isotropic noise, and we develop tools for Gaussian-like as well as heavy-tail types of noise in particular. As a main contribution, we derive th…
Wireless channel sensing is one of the key enablers for integrated sensing and communication (ISAC) which helps communication networks understand the surrounding environment. In this work, we consider MIMO-OFDM systems and aim to design optimal and robust waveforms for accurate channel parameter estimation given allocated OFDM resources. The Fisher information matrix (FIM) is derived first, and the waveform design problem is formulated by maximizing the log determinant of the FIM. We then consider the uncertainty in the parameters and state the stochastic optimization problem for a robust desi…
Mobile communication networks were designed to mainly support ubiquitous wireless communications, yet they are also expected to achieve radio sensing capabilities in the near future. However, most prior studies on radio sensing usually rely on far-field assumption with uniform plane wave (UPW) models. With the ever-increasing antenna size, together with the growing demands to sense nearby targets, the conventional far-field UPW assumption may become invalid. Therefore, this paper studies near-field radio sensing with extremely large-scale (XL) antenna arrays, where the more general uniform sph…
We seek to characterize the estimation performance of a sensor network where the individual sensors exhibit the phenomenon of drift, i.e., a gradual change of the bias. Though estimation in the presence of random errors has been extensively studied in the literature, the loss of estimation performance due to systematic errors like drift have rarely been looked into. In this paper, we derive closed-form Fisher Information matrix and subsequently Cramér-Rao bounds (upto reasonable approximation) for the estimation accuracy of drift-corrupted signals. We assume a polynomial time-series as the rep…
The constrained Cramer-Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood (CML) estimator, in the non-asymptotic region these conditions are usually too strict and the commonly-used estimators, such as the CML estimator, do not satisfy them. Therefore, the CCRB may not be a lower bound on the MSE matrix of such estimators. In this paper, we propose a new definition for unbiasedness under constraints, denoted by C-un…
Tsinghua University（清华大学）；University of Southern California（南加利福尼亚大学）
Channel state information (CSI) is of vital importance in wireless communication systems. Existing CSI acquisition methods usually rely on pilot transmissions, and geographically separated base stations (BSs) with non-correlated CSI need to be assigned with orthogonal pilots which occupy excessive system resources. Our previous work adopts a data-driven deep learning based approach which leverages the CSI at a local BS to infer the CSI remotely, however the relevance of CSI between separated BSs is not specified explicitly. In this paper, we exploit a model-based methodology to derive the Cram…
