It is well known that any positive matrix can be scaled to have prescribed row and column sums by multiplying its rows and columns by certain positive scaling factors (which are unique up to a positive scalar). This procedure is known as matrix scaling, and has found numerous applications in operations research, economics, image processing, and machine learning. In this work, we investigate the behavior of the scaling factors and the resulting scaled matrix when the matrix to be scaled is random. Specifically, letting $\widetilde{A}\in\mathbb{R}^{M\times N}$ be a positive and bounded random matrix whose entries assume a certain type of independence, we provide a concentration inequality for the scaling factors of $\widetilde{A}$ around those of $A = \mathbb{E}[\widetilde{A}]$. This result is employed to bound the convergence rate of the scaling factors of $\widetilde{A}$ to those of $A$, as well as the concentration of the scaled version of $\widetilde{A}$ around the scaled version of $A$ in operator norm, as $M,N\rightarrow\infty$. When the entries of $\widetilde{A}$ are independent, $M=N$, and all prescribed row and column sums are $1$ (i.e., doubly-stochastic matrix scaling), both of the previously-mentioned bounds are $\mathcal{O}(\sqrt{\log N / N})$ with high probability. We demonstrate our results in several simulations.
We present a concentration inequality for linear functionals of noncommutative polynomials in random matrices. Our hypotheses cover most standard ensembles, including Gaussian matrices, matrices with independent uniformly bounded entries and unitary or orthogonal matrices.
It is well-known that distances in random iid matrices are highly concentrated around their mean. In this note we extend this concentration phenomenon to Wigner matrices. Exponential bounds for the lower tail are also included.
We derive concentration inequalities for functions of the empirical measure of large random matrices with infinitely divisible entries and, in particular, stable ones. We also give concentration results for some other functionals of these random matrices, such as the largest eigenvalue or the largest singular value.
The present work provides an original framework for random matrix analysis based on revisiting the concentration of measure theory from a probabilistic point of view. By providing various notions of vector concentration ($q$-exponential, linear, Lipschitz, convex), a set of elementary tools is laid out that allows for the immediate extension of classical results from random matrix theory involving random concentrated vectors in place of vectors with independent entries. These findings are exemplified here in the context of sample covariance matrices but find a large range of applications in st…
Given a finite group, we study the Gaussian series of the matrices in the image of its left regular representation. We propose such random matrices as a benchmark for improvements to the noncommutative Khintchine inequality, and we highlight an application to the matrix Spencer conjecture.
We prove that few largest (and most important) eigenvalues of random symmetric matrices of various kinds are very strongly concentrated. This strong concentration enables us to compute the means of these eigenvalues with high precision. Our approach uses Talagrand's inequality and is very different from standard approaches.
We prove concentration results for $\ell_p^n$ operator norms of rectangular random matrices and eigenvalues of self-adjoint random matrices. The random matrices we consider have bounded entries which are independent, up to a possible self-adjointness constraint. Our results are based on an isoperimetric inequality for product spaces due to Talagrand.
We show that the mixing times of random walks on compact groups can be used to obtain concentration inequalities for the respective Haar measures. As an application, we derive a concentration inequality for the empirical distribution of eigenvalues of sums of random hermitian matrices, with possible applications in free probability. The advantage over existing techniques is that the new method can deal with functions that are non-Lipschitz or even discontinuous with respect to the usual metrics.
Matrix concentration inequalities provide a direct way to bound the typical spectral norm of a random matrix. The methods for establishing these results often parallel classical arguments, such as the Laplace transform method. This work develops a matrix extension of the entropy method, and it applies these ideas to obtain some matrix concentration inequalities.
Carnegie Mellon University（卡内基梅隆大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
The spectra of random feature matrices provide essential information on the conditioning of the linear system used in random feature regression problems and are thus connected to the consistency and generalization of random feature models. Random feature matrices are asymmetric rectangular nonlinear matrices depending on two input variables, the data and the weights, which can make their characterization challenging. We consider two settings for the two input variables, either both are random variables or one is a random variable and the other is well-separated, i.e. there is a minimum distanc…
In this short note we derive concentration inequalities for the empirical absolute moments of square symmetric matrices with independent symmetrically distributed +/-1 entries. Most of the previous results of this type are limited to functions with bounded Lipschitz constant, and therefore exclude the moments from consideration. Based on the fine analysis of the distribution of the largest eigenvalues developed by Soshnikov and Sinai, we extend the measure concentration tools of Talagrand to encompass power functions.
We investigate concentration properties of spectral measures of Hermitian random matrices with partially dependent entries. More precisely, let $X_n$ be a Hermitian random matrix of size $n\times n$ that can be split into independent blocks of the size at most $d_n=o(n^2)$. We prove that under some mild conditions on the distribution of the entries of $X_n$, the empirical spectral measure of $X_n$ concentrates around its mean. The main theorem is a strengthening of a recent result by Kemp and Zimmerman, where the size of blocks grows as $o(\log n)$. As an application, we are able to upgrade th…
The topic of this paper is the typical behavior of the spectral measures of large random matrices drawn from several ensembles of interest, including in particular matrices drawn from Haar measure on the classical Lie groups, random compressions of random Hermitian matrices, and the so-called random sum of two independent random matrices. In each case, we estimate the expected Wasserstein distance from the empirical spectral measure to a deterministic reference measure, and prove a concentration result for that distance. As a consequence we obtain almost sure convergence of the empirical spect…
We place ourselves in the setting of high-dimensional statistical inference, where the number of variables $p$ in a data set of interest is of the same order of magnitude as the number of observations $n$. More formally, we study the asymptotic properties of correlation and covariance matrices, in the setting where $p/n\toρ\in(0,\infty),$ for general population covariance. We show that, for a large class of models studied in random matrix theory, spectral properties of large-dimensional correlation matrices are similar to those of large-dimensional covarance matrices. We also derive a Marucenk…
The circular law asserts that if $X_n$ is a $n \times n$ matrix with iid complex entries of mean zero and unit variance, then the empirical spectral distribution of $\frac{1}{\sqrt{n}} X_n$ converges almost surely to the uniform distribution on the unit disk as $n$ tends to infinity. Answering a question of Tao, we prove the circular law for a general class of random block matrices with dependent entries. The proof relies on an inverse-type result for the concentration of linear operators and multilinear forms.
Let A_n=(a_{ij})_{i,j=1}^n be an n\times n positive matrix with entries in [a,b], 0<a\le b. Let X_n=(\sqrta_{ij}x_{ij})_{i,j=1}^n be a random matrix, where {x_{ij}} are i.i.d. N(0,1) random variables. We show that for large n, \det (X_n^TX_n) concentrates sharply at the permanent of A_n, in the sense that n^{-1}\log (\det(X_n^TX_n)/perA_n)\to_{n\to\infty}0 in probability.
Consider $n$ complex random matrices $X_1,\ldots,X_n$ of size $d\times d$ sampled i.i.d. from a distribution with mean $E[X]=μ$. While the concentration of averages of these matrices is well-studied, the concentration of other functions of such matrices is less clear. One function which arises in the context of stochastic iterative algorithms, like Oja's algorithm for Principal Component Analysis, is the normalized matrix product defined as $\prod\limits_{i=1}^{n}\left(I + \frac{X_i}{n}\right).$ Concentration properties of this normalized matrix product were recently studied by \cite{HW19}. Ho…
This paper develops nonasymptotic growth and concentration bounds for a product of independent random matrices. These results sharpen and generalize recent work of Henriksen-Ward, and they are similar in spirit to the results of Ahlswede-Winter and of Tropp for a sum of independent random matrices. The argument relies on the uniform smoothness properties of the Schatten trace classes.
Let $W_n= \frac{1}{\sqrt n} M_n$ be a Wigner matrix whose entries have vanishing third moment, normalized so that the spectrum is concentrated in the interval $[-2,2]$. We prove a concentration bound for $N_I = N_I(W_n)$, the number of eigenvalues of $W_n$ in an interval $I$. Our result shows that $N_I$ decays exponentially with standard deviation at most $O(\log^{O(1)} n)$. This is best possible up to the constant exponent in the logarithmic term. As a corollary, the bulk eigenvalues are localized to an interval of width $O(\log^{O(1)} n/n)$; again, this is optimal up to the exponent. These r…
University of Chicago（芝加哥大学）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究院）
Analyzing concentration of large random matrices is a common task in a wide variety of fields. Given independent random variables, many tools are available to analyze random matrices whose entries are linear in the variables, e.g. the matrix-Bernstein inequality. However, in many applications, we need to analyze random matrices whose entries are polynomials in the variables. These arise naturally in the analysis of spectral algorithms, e.g., Hopkins et al. [STOC 2016], Moitra-Wein [STOC 2019]; and in lower bounds for semidefinite programs based on the Sum of Squares hierarchy, e.g. Barak et al…
Talagrand's inequalities make a link between two fundamentals concepts of probability: transport of measures and entropy. The study of the counterpart of these inequalities in the context of free probability has been initiated by Biane and Voiculescu and later extended by Hiai, Petz and Ueda for convex potentials. In this work, we prove a free analogue of a result of Bobkov and Götze in the classical setting, thus providing free transport-entropy inequalities for a very natural class of measures appearing in random matrix theory. These inequalities are weaker than the ones of Hiai, Petz and Ue…
Let H=A+UBU* where A and B are two N-by-N Hermitian matrices and U is a Haar-distributed random unitary matrix, and let μ_H, μ_A, and μ_B be empirical measures of eigenvalues of matrices H, A, and B, respectively. Then, it is known (see, for example, Pastur-Vasilchuk, CMP, 2000, v.214, pp.249-286) that for large N, measure μ_H is close to the free convolution of measures μ_A and μ_B, where the free convolution is a non-linear operation on probability measures. The large deviations of the cumulative distribution function of μ_H from its expectation have been studied by Chatterjee in in JFA, 200…
Computing the distribution of permanents of random matrices has been an outstanding open problem for several decades. In quantum computing, "anti-concentration" of this distribution is an unproven input for the proof of hardness of the task of boson-sampling. We study permanents of random i.i.d. complex Gaussian matrices, and more broadly, submatrices of random unitary matrices. Using a hybrid representation-theoretic and combinatorial approach, we prove strong lower bounds for all moments of the permanent distribution. We provide substantial evidence that our bounds are close to being tight a…
Many problems in combinatorial linear algebra require upper bounds on the number of solutions to an underdetermined system of linear equations $Ax = b$, where the coordinates of the vector $x$ are restricted to take values in some small subset (e.g. $\{\pm 1\}$) of the underlying field. The classical ways of bounding this quantity are to use either a rank bound observation due to Odlyzko or a vector anti-concentration inequality due to Halász. The former gives a stronger conclusion except when the number of equations is significantly smaller than the number of variables; even in such situation…
We define a "renormalized energy" as an explicit functional on arbitrary point configurations of constant average density in the plane and on the real line. The definition is inspired by ideas of [SS1,SS3]. Roughly speaking, it is obtained by subtracting two leading terms from the Coulomb potential on a growing number of charges. The functional is expected to be a good measure of disorder of a configuration of points. We give certain formulas for its expectation for general stationary random point processes. For the random matrix $β$-sine processes on the real line (beta=1,2,4), and Ginibre po…
Let ${\mathcal D}_{n,d}$ be the set of all $d$-regular directed graphs on $n$ vertices. Let $G$ be a graph chosen uniformly at random from ${\mathcal D}_{n,d}$ and $M$ be its adjacency matrix. We show that $M$ is invertible with probability at least $1-C\ln^{3} d/\sqrt{d}$ for $C\leq d\leq cn/\ln^2 n$, where $c, C$ are positive absolute constants. To this end, we establish a few properties of $d$-regular directed graphs. One of them, a Littlewood-Offord type anti-concentration property, is of independent interest. Let $J$ be a subset of vertices of $G$ with $|J|\approx n/d$. Let $δ_i$ be the i…
Let $X,X_1,...,X_n$ be independent identically distributed random variables. In this paper we study the behavior of the concentration functions of the weighted sums $\sum\limits_{k=1}^{n}a_k X_k$ with respect to the arithmetic structure of coefficients $a_k$. Such concentration results recently became important in connection with investigations about singular values of random matrices. In this paper we formulate and prove some refinements of a result of Vershynin (R. Vershynin, Invertibility of symmetric random matrices, arXiv:1102.0300. (2011). Published in Random Structures and Algorithms, v…
Slepian and Sudakov-Fernique type inequalities, which compare expectations of maxima of Gaussian random vectors under certain restrictions on the covariance matrices, play an important role in probability theory, especially in empirical process and extreme value theories. Here we give explicit comparisons of expectations of smooth functions and distribution functions of maxima of Gaussian random vectors without any restriction on the covariance matrices. We also establish an anti-concentration inequality for the maximum of a Gaussian random vector, which derives a useful upper bound on the Lév…
This paper derives exponential concentration inequalities and polynomial moment inequalities for the spectral norm of a random matrix. The analysis requires a matrix extension of the scalar concentration theory developed by Sourav Chatterjee using Stein's method of exchangeable pairs. When applied to a sum of independent random matrices, this approach yields matrix generalizations of the classical inequalities due to Hoeffding, Bernstein, Khintchine and Rosenthal. The same technique delivers bounds for sums of dependent random matrices and more general matrix-valued functions of dependent rand…
This paper derives exponential tail bounds and polynomial moment inequalities for the spectral norm deviation of a random matrix from its mean value. The argument depends on a matrix extension of Stein's method of exchangeable pairs for concentration of measure, as introduced by Chatterjee. Recent work of Mackey et al. uses these techniques to analyze random matrices with additive structure, while the enhancements in this paper cover a wider class of matrix-valued random elements. In particular, these ideas lead to a bounded differences inequality that applies to random matrices constructed fr…
In this expository note, we give a modern proof of Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. We deduce a useful concentration inequality for sub-gaussian random vectors. Two examples are given to illustrate these results: a concentration of distances between random vectors and subspaces, and a bound on the norms of products of random and deterministic matrices.
To a $N \times N$ real symmetric matrix Kerov assigns a piecewise linear function whose local minima are the eigenvalues of this matrix and whose local maxima are the eigenvalues of its $(N-1) \times (N-1)$ submatrix. We study the scaling limit of Kerov's piecewise linear functions for Wigner and Wishart matrices. For Wigner matrices the scaling limit is given by the Verhik-Kerov-Logan-Shepp curve which is known from asymptotic representation theory. For Wishart matrices the scaling limit is also explicitly found, and we explain its relation to the Marchenko-Pastur limit spectral law.
Higher-order spacing ratios are investigated analytically using a Wigner-like surmise for Gaussian ensembles of random matrices. For $k$-th order spacing ratio $(r^{(k)}$, $k>1)$ the matrix of dimension $2k+1$ is considered. A universal scaling relation for this ratio, known from earlier numerical studies, is proved in the asymptotic limits of $r^{(k)}\rightarrow0$ and $r^{(k)}\rightarrow \infty$.
In this short note, we extend the celebrated results of Tao and Vu, and Krishnapur on the universality of empirical spectral distributions to a wide class of inhomogeneous complex random matrices, by showing that a technical and hard-to-verify Fourier domination assumption may be replaced simply by a natural uniform anti-concentration assumption. Along the way, we show that inhomogeneous complex random matrices, whose expected squared Hilbert-Schmidt norm is quadratic in the dimension, and whose entries (after symmetrization) are uniformly anti-concentrated at $0$ and infinity, typically have …
This paper establishes new concentration inequalities for random matrices constructed from independent random variables. These results are analogous with the generalized Efron-Stein inequalities developed by Boucheron et al. The proofs rely on the method of exchangeable pairs.
The concentration of measure phenomenon may be summarized as follows: a function of many weakly dependent random variables that is not too sensitive to any of its individual arguments will tend to take values very close to its expectation. This phenomenon is most completely understood when the arguments are mutually independent random variables, and there exist several powerful complementary methods for proving concentration inequalities, such as the martingale method, the entropy method, and the method of transportation inequalities. The setting of dependent arguments is much less well unders…
We present some results concerning the almost sure behaviour of the operator norm or random Toeplitz matrices, including the law of large numbers for the norm, normalized by its expectation (in the i.i.d. case). As tools we present some concentration inequalities for suprema of empirical processes, which are refinements of recent results by Einmahl and Li.
We give a new, elementary proof of a key inequality used by Rudelson in the derivation of his well-known bound for random sums of rank-one operators. Our approach is based on Ahlswede and Winter's technique for proving operator Chernoff bounds. We also prove a concentration inequality for sums of random matrices of rank one with explicit constants.
Suppose $\{ X_k \}_{k \in \mathbb{Z}}$ is a sequence of bounded independent random matrices with common dimension $d\times d$ and common expectation $\mathbb{E}[ X_k ]= X$. Under these general assumptions, the normalized random matrix product $$Z_n = (I + \frac{1}{n}X_n)(I + \frac{1}{n}X_{n-1}) \cdots (I + \frac{1}{n}X_1)$$ converges to $Z_n \rightarrow e^{X}$ as $n \rightarrow \infty$. Normalized random matrix products of this form arise naturally in stochastic iterative algorithms, such as Oja's algorithm for streaming Principal Component Analysis. Here, we derive nonasymptotic concentration…
Fisher-KPP equation is proved to be the scaling limit of a system of Brownian particles with local interaction. Particles proliferate and die depending on the local concentration of other particles. Opposite to discrete models, controlling concentration of particles is a major difficulty in Brownian particle interaction; local interactions instead of mean field or moderate ones makes it more difficult to implement the law of large numbers properties. The approach taken here to overcome these difficulties is largely inspired by A. Hammond and F. Rezakhanlou [10] implemented there in the mean fr…
Akemann, Ipsen and Kieburg recently showed that the squared singular values of products of M rectangular random matrices with independent complex Gaussian entries are distributed according to a determinantal point process with a correlation kernel that can be expressed in terms of Meijer G-functions. We show that this point process can be interpreted as a multiple orthogonal polynomial ensemble. We give integral representations for the relevant multiple orthogonal polynomials and a new double contour integral for the correlation kernel, which allows us to find its scaling limits at the origin …
We study asymptotics of the Itzykson-Zuber integrals in the scaling when one of the matrices has a small rank compared to the full rank. We show that the result is basically the same as in the case when one of the matrices has a fixed rank. In this way we extend the recent results of Guionnet and Maida who showed that for a latter scaling the Itzykson-Zuber integral is given in terms of the Voiculescu's R-transform of the full rank matrix.
