Let A_n=(a_{ij})_{i,j=1}^n be an n\times n positive matrix with entries in [a,b], 0<a\le b. Let X_n=(\sqrta_{ij}x_{ij})_{i,j=1}^n be a random matrix, where {x_{ij}} are i.i.d. N(0,1) random variables. We show that for large n, \det (X_n^TX_n) concentrates sharply at the permanent of A_n, in the sense that n^{-1}\log (\det(X_n^TX_n)/perA_n)\to_{n\to\infty}0 in probability.
We show that the absolute value of the determinant of a matrix with random independent (but not necessarily iid) entries is strongly concentrated around its mean. As an application, we show that the Godsil-Gutman and Barvinok estimators for the permanent of a strictly positive matrix give sub-exponential approximation ratios with high probability.
We present estimates on the small singular values of a class of matrices with independent Gaussian entries and inhomogeneous variance profile, satisfying a broad-connectedness condition. Using these estimates and concentration of measure for the spectrum of Gaussian matrices with independent entries, we prove that for a large class of graphs satisfying an appropriate expansion property, the Barvinok--Godsil-Gutman estimator for the permanent achieves sub-exponential errors with high probability.
Computing the distribution of permanents of random matrices has been an outstanding open problem for several decades. In quantum computing, "anti-concentration" of this distribution is an unproven input for the proof of hardness of the task of boson-sampling. We study permanents of random i.i.d. complex Gaussian matrices, and more broadly, submatrices of random unitary matrices. Using a hybrid representation-theoretic and combinatorial approach, we prove strong lower bounds for all moments of the permanent distribution. We provide substantial evidence that our bounds are close to being tight a…
We introduce a new notion of the determinant, called symmetrized determinant, for a square matrix with the entries in an associative algebra $Å$. The monomial expansion of the symmetrized determinant is obtained from the standard expansion of the commutative determinant by averaging the products of entries of the matrix in all possible orders. We show that for any fixed finite-dimensional associative algebra $Å$, the symmetrized determinant of an $n\times n$ matrix with the entries in $Å$ can be computed in polynomial in $n$ time (the degree of the polynomial is linear in the dimension of $Å$)…
Permanents of random matrices with independent and identically distributed (i.i.d.) entries have extensively studied in literature and convergence and concentration properties are known under varying assumptions on the distributions. In this paper we consider constrained~\(n \times n\) random matrices where each row has a deterministic number~\(r = r(n)\) zero entries and the rest of the entries are independent random variables that are positive almost surely. The positions of the zeros within each row are also random and we establish sufficient conditions for the existence of a weak law of la…
We prove a strong concentration result about the natural collision estimator, which counts the number of collisions that occur within an iid sample. This estimator is at the heart of algorithms used for uniformity testing and entropy assessment. While the prior works were limited to only variance, we use elegant techniques of independent interest to bounds higher moments and conclude concentration properties. As an immediate corollary we show that the estimator achieves high-probability guarantee on its own and there is no need for boosting (aka median/majority trick).
In this work we design a general method for proving moment inequalities for polynomials of independent random variables. Our method works for a wide range of random variables including Gaussian, Boolean, exponential, Poisson and many others. We apply our method to derive general concentration inequalities for polynomials of independent random variables. We show that our method implies concentration inequalities for some previously open problems, e.g. permanent of a random symmetric matrices. We show that our concentration inequality is stronger than the well-known concentration inequality due …
Covariance estimation becomes challenging in the regime where the number p of variables outstrips the number n of samples available to construct the estimate. One way to circumvent this problem is to assume that the covariance matrix is nearly sparse and to focus on estimating only the significant entries. To analyze this approach, Levina and Vershynin (2011) introduce a formalism called masked covariance estimation, where each entry of the sample covariance estimator is reweighted to reflect an a priori assessment of its importance. This paper provides a short analysis of the masked sample co…
Our goal is to establish large deviations and concentration inequalities for the maximum likelihood estimator of the drift parameter of the Ornstein-Uhlenbeck process without tears. We propose a new strategy to establish large deviation results which allows us, via a suitable transformation, to circumvent the classical difficulty of non-steepness. Our approach holds in the stable case where the process is positive recurrent as well as in the unstable and explosive cases where the process is respectively null recurrent and transient. Notwithstanding of this trichotomy, we also provide new conce…
We derive tight and computable bounds on the bias of statistical estimators, or more generally of quantities of interest, when evaluated on a baseline model P rather than on the typically unknown true model Q. Our proposed method combines the scalable information inequality derived by P. Dupuis, K.Chowdhary, the authors and their collaborators together with classical concentration inequalities (such as Bennett's and Hoeffding-Azuma inequalities). Our bounds are expressed in terms of the Kullback-Leibler divergence R(Q||P) of model Q with respect to P and the moment generating function for the …
We obtain bounds on fluctuations of two entropy estimators for a class of one-dimensional Gibbs measures on the full shift. They are the consequence of a general exponential inequality for Lipschitz functions of n variables. The first estimator is based on empirical frequencies of blocks scaling logarithmically with the sample length. The second one is based on the first appearance of blocks within typical samples.
Optimal concentration and gradient sensing by perfect monitors
Institute for Theoretical Physics, Department of Physics, Utrecht University(乌得勒支大学理论物理系);Centre for Complex Systems Studies, Utrecht University(乌得勒支大学复杂系统研究中心)
This article studies the \emph{robust covariance matrix estimation} of a data collection $X = (x_1,\ldots,x_n)$ with $x_i = \sqrt τ_i z_i + m$, where $z_i \in \mathbb R^p$ is a \textit{concentrated vector} (e.g., an elliptical random vector), $m\in \mathbb R^p$ a deterministic signal and $τ_i\in \mathbb R$ a scalar perturbation of possibly large amplitude, under the assumption where both $n$ and $p$ are large. This estimator is defined as the fixed point of a function which we show is contracting for a so-called \textit{stable semi-metric}. We exploit this semi-metric along with concentration …
We study the estimation and concentration on its expectation of the probability to observe data further than a specified distance from a given iid sample in a metric space. The problem extends the classical problem of estimation of the missing mass in discrete spaces. We give some estimators for the conditional missing mass and show that estimation of the expected missing mass is difficult in general. Conditions on the distribution, under which the Good-Turing estimator and the conditional missing mass concentrate on their expectations are identified. Applications to anomaly detection, coding,…
We derive uniform concentration inequalities for continuous-time analogues of empirical processes and related stochastic integrals of scalar ergodic diffusion processes. Thereby, we lay the foundation typically required for the study of sup-norm properties of estimation procedures for a large class of diffusion processes. In the classical i.i.d. context, a key device for the statistical sup-norm analysis is provided by Talagrand-type concentration inequalities. Aiming for a parallel substitute in the diffusion framework, we present a systematic, self-contained approach to such uniform concentr…
Constant-specified and exponential concentration inequalities play an essential role in the finite-sample theory of machine learning and high-dimensional statistics area. We obtain sharper and constants-specified concentration inequalities for the sum of independent sub-Weibull random variables, which leads to a mixture of two tails: sub-Gaussian for small deviations and sub-Weibull for large deviations from the mean. These bounds are new and improve existing bounds with sharper constants. In addition, a new sub-Weibull parameter if the italic should be retained. Please check the whole text. i…
A UCB Bandit Algorithm for General ML-Based Estimators
School of Mathematics, University of Minnesota—Twin Cities(明尼苏达大学 Twin Cities 分校数学系);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系)
We consider a finite impulse response system with centered independent sub-Gaussian design covariates and noise components that are not necessarily identically distributed. We derive non-asymptotic near-optimal estimation and prediction bounds for the least-squares estimator of the parameters. Our results are based on two concentration inequalities on the norm of sums of dependent covariate vectors and on the singular values of their covariance operator that are of independent value on their own and where the dependence arises from the time shift structure of the time series. These results gen…
Network-valued time series are currently a common form of network data. However, the study of the aggregate behavior of network sequences generated from network-valued stochastic processes is relatively rare. Most of the existing research focuses on the simple setup where the networks are independent (or conditionally independent) across time, and all edges are updated synchronously at each time step. In this paper, we study the concentration properties of the aggregated adjacency matrix and the corresponding Laplacian matrix associated with network sequences generated from lazy network-valued…