This paper investigates a statistical procedure for testing the equality of two independent estimated covariance matrices when the number of potentially dependent data vectors is large and proportional to the size of the vectors, that is, the number of variables. Inspired by the spike models used in random matrix theory, we concentrate on the largest eigenvalues of the matrices in order to determine significance. To avoid false rejections we must guard against residual spikes and need a sufficiently precise description of the behaviour of the largest eigenvalues under the null hypothesis.
In this paper we propose some "invariant" theorems that allows us to extend the test of arXiv:2002.12741 for perturbation of order $1$ to some general tests for order $k$. The statistics introduced in this paper allow the user to test the equality of two populations based on high-dimensional multivariate data.
Simulations show that these tests have more power of detection than standard multivariate approaches.
This paper investigates a statistical procedure for testing the equality of two independent estimated covariance matrices when the number of potentially dependent data vectors is large and proportional to the size of the vectors, that is, the number of variables. Inspired by the spike models used in random matrix theory, we concentrate on the largest eigenvalues of the matrices in order to determine significance. To avoid false rejections we must guard against residual spikes and need a sufficiently precise description of the behaviour of the largest eigenvalues under the null hypothesis. In t…
Motivated by population studies of Diffusion Tensor Imaging, the paper investigates the use of mean-based and dispersion-based permutation tests to define and compute the significance of a statistical test for data taking values on nonlinear manifolds. The paper proposes statistical tests that are computationally tractable and geometrically sound for Diffusion Tensor Imaging.
Different network structures are compiared with respect to degree of robustnes of identification statistical procedures. It is shown that threshold (market) graph, cliques and independent sets in the threshold (market) graphs are preferable network structure from this point of view.
We consider a binary branching process structured by a stochastic trait that evolves according to a diffusion process that triggers the branching events, in the spirit of Kimmel's model of cell division with parasite infection. Based on the observation of the trait at birth of the first n generations of the process, we construct nonparametric estimator of the transition of the associated bifurcating chain and study the parametric estimation of the branching rate. In the limit, as n tends to infinity, we obtain asymptotic efficiency in the parametric case and minimax optimality in the nonparame…
We recall the main concepts of the Le Cam theory of statistical experiments , especially the notion of Le Cam distance and its properties. We also review classical tools for bounding such a distance before presenting some examples. A proof of the classical equivalence result between density estimation problems and Gaussian white noise models will be analyzed.
Modern statistical inference tasks often require iterative optimization methods to compute the solution. Convergence analysis from an optimization viewpoint only informs us how well the solution is approximated numerically but overlooks the sampling nature of the data. In contrast, recognizing the randomness in the data, statisticians are keen to provide uncertainty quantification, or confidence, for the solution obtained using iterative optimization methods. This paper makes progress along this direction by introducing the moment-adjusted stochastic gradient descents, a new stochastic optimiz…
Statistical pragmatism embraces all efficient methods in statistical inference. Augmentation of the collected data is used herein to obtain representative population information from a large class of non-representative population's units. Parameter expansion of a probability model is shown to reduce the upper bound on the sum of error probabilities for a test of simple hypotheses, and a measure, R, is proposed for the effect of activating additional component(s) in the sufficient statistic.
General hypergeometric distribution (GHGD) describes the following distribution: from a finite space containing N elements, select T subsets with each subset contains M[i] (T-1 >= i >= 0) elements, what is the probability that exactly x elements are overlapped exactly t times or at least t times (XLO=t or XLO>=t, T >= t >= 0, here LO is level of overlap)? The classical hypergeometric distribution (HGD) describes the situation of two subsets, while the general situation has not been resolved, despite the overlapped elements has been visualized with the Venn diagram method for about 140 years. G…
We study the statistical and computational complexities of the Polyak step size gradient descent algorithm under generalized smoothness and Lojasiewicz conditions of the population loss function, namely, the limit of the empirical loss function when the sample size goes to infinity, and the stability between the gradients of the empirical and population loss functions, namely, the polynomial growth on the concentration bound between the gradients of sample and population loss functions. We demonstrate that the Polyak step size gradient descent iterates reach a final statistical radius of conve…
Image fusion produces a single fused image from a set of input images. A new method for image fusion is proposed based on Weighted Average Merging Method (WAMM) in the NonSubsampled Contourlet Transform (NSCT) domain. A performance analysis on various statistical fusion rules are also analysed both in NSCT and Wavelet domain. Analysis has been made on medical images, remote sensing images and multi focus images. Experimental results shows that the proposed method, WAMM obtained better results in NSCT domain than the wavelet domain as it preserves more edges and keeps the visual quality intact …
General linear models (GLM) are often constructed and used in statistical inference at the voxel level in brain imaging. In this paper, we explore the basics of random fields and the multiple comparisons on the random fields, which are necessary to properly threshold statistical maps for the whole image at specific statistical significance level. The multiple comparisons are crucial in determining overall statistical significance in correlated test statistics over the whole brain. In practice, t- or F-statistics in adjacent voxels are correlated. So there is the problem of multiple comparisons…
We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to updates based on a finite set of samples. First, we characterize the domain of attraction of any global maximizer of the population likelihood. This characterization is based on a novel view of the EM updates as a perturbed form of likelihood ascent, or in parallel, of the gradient EM updates as a …
A family of probability distributions (i.e. a statistical model) is said to be sufficient for another, if there exists a transition matrix transforming the probability distributions in the former to the probability distributions in the latter. The Blackwell-Sherman-Stein (BSS) theorem provides necessary and sufficient conditions for one statistical model to be sufficient for another, by comparing their information values in statistical decision problems. In this paper we extend the BSS theorem to quantum statistical decision theory, where statistical models are replaced by families of density …
