Concentration Bounds for the Collision Estimator

How biased is your model? Concentration Inequalities, Information and Model Bias

We derive tight and computable bounds on the bias of statistical estimators, or more generally of quantities of interest, when evaluated on a baseline model P rather than on the typically unknown true model Q. Our proposed method combines the scalable information inequality derived by P. Dupuis, K.Chowdhary, the authors and their collaborators together with classical concentration inequalities (such as Bennett's and Hoeffding-Azuma inequalities). Our bounds are expressed in terms of the Kullback-Leibler divergence R(Q||P) of model Q with respect to P and the moment generating function for the …

Exponential Concentration of a Density Functional Estimator

We analyze a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the $d$-dimensional unit cube $[0,1]^d$ that lie in a $β$-Hölder smoothness class, we prove our estimator converges at the rate $O \left( n^{-\fracβ{β+ d}} \right)$. Furthermore, we prove the estimator is exponentially concentrated about its mean, whereas most previous related results have proven only expected error bounds on estimators.

超指向性作为谱碰撞再生核希尔伯特空间极限

Superdirectivity as Boundary Concentration under Spectral Collision

本文通过谱碰撞极限和多项式射流几何，在再生核希尔伯特空间框架下解释阵列超指向性，发现端射方向增益的M²定律源于Christoffel-Darboux核的端点渐近行为，并指出超指向性是几何边界集中现象。

最大似然估计的次高斯集中与熵正态性

Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator

本文在标准正则条件下，通过建立归一化最大似然估计的次高斯尾界、所有矩收敛以及熵中心极限定理，强化了经典渐近正态性结果。

Generalized Exponential Concentration Inequality for Rényi Divergence Estimation

Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Rényi-$α$ divergence for a smooth Hölder class of densities on the $d$-dimensional unit cube $[0, 1]^d$. We also illustrate our theoretical results with a numerical experiment.

Generalized LRS Estimator for Min-entropy Estimation

The min-entropy is a widely used metric to quantify the randomness of generated random numbers, which measures the difficulty of guessing the most likely output. It is difficult to accurately estimate the min-entropy of a non-independent and identically distributed (non-IID) source. Hence, NIST Special Publication (SP) 800-90B adopts ten different min-entropy estimators and then conservatively selects the minimum value among ten min-entropy estimates. Among these estimators, the longest repeated substring (LRS) estimator estimates the collision entropy instead of the min-entropy by counting th…