Asymptotic distribution for the proportional covariance model under multivariate normal distributions is derived. To this end, the parametrization of the common covariance matrix by its Cholesky root is adopted. The derivations are made in three steps. First, the asymptotic distribution of the maximum likelihood estimators of the proportionality coefficients and the Cholesky inverse root of the common covariance matrix is derived by finding the information matrix and its inverse. Next, the asymptotic distributions for the case of the Cholesky root of the common covariance matrix and finally for the case of the common covariance matrix itself are derived using the multivariate $δ$-method. As an application of the asymptotic distribution derived here, a hypothesis for homogeneity of covariance matrices is considered.
The Behrens-Fisher problem is a well-known hypothesis testing problem in statistics concerning two-sample mean comparison. In this article, we confirm one conjecture in Eaton and Olshen (1972), which provides stochastic bounds for the multivariate Behrens-Fisher test statistic under the null hypothesis. We also extend their results on the stochastic ordering of random quotients to the arbitrary finite dimensional case. This work can also be seen as a generalization of Hsu (1938) that provided the bounds for the univariate Behrens-Fisher problem. The results obtained in this article can be used…
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
This article provides an introduction to the asymptotic analysis of covariance parameter estimation for Gaussian processes. Maximum likelihood estimation is considered. The aim of this introduction is to be accessible to a wide audience and to present some existing results and proof techniques from the literature. The increasing-domain and fixed-domain asymptotic settings are considered. Under increasing-domain asymptotics, it is shown that in general all the components of the covariance parameter can be estimated consistently by maximum likelihood and that asymptotic normality holds. In contr…
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
Due to the increasing recording capability, functional data analysis has become an important research topic. For functional data the study of outlier detection and/or the development of robust statistical procedures has started recently. One robust alternative to the sample covariance operator is the sample spatial sign covariance operator. In this paper, we study the asymptotic behaviour of the sample spatial sign covariance operator when location is unknown. Among other possible applications of the obtained results, we derive the asymptotic distribution of the principal directions obtained f…
We study estimation and prediction of Gaussian random fields with covariance models belonging to the generalized Wendland (GW) class, under fixed domain asymptotics. As the Matérn case, this class allows a continuous parameterization of smoothness of the underlying Gaussian random field, being additionally compactly supported. The paper is divided into two parts: First, we characterize the equivalence of two Gaussian measures with GW covariance function, and we provide sufficient conditions for the equivalence of two Gaussian measures with Matérn and GW covariance functions. We elucidate the c…
In arXiv:0907.0079 by Cator and Lopuhaa, an asymptotic expansion for the MCD estimators is established in a very general framework. This expansion requires the existence and non-singularity of the derivative in a first-order Taylor expansion. In this paper, we prove the existence of this derivative for multivariate distributions that have a density and provide an explicit expression. Moreover, under suitable symmetry conditions on the density, we show that this derivative is non-singular. These symmetry conditions include the elliptically contoured multivariate location-scatter model, in which…
We consider a one-dimensional Gaussian process having exponential covariance function. Under fixed-domain asymptotics, we prove the strong consistency and asymptotic normality of a cross validation estimator of the microergodic covariance parameter. In this setting, Ying [40] proved the same asymptotic properties for the maximum likelihood estimator. Our proof includes several original or more involved components, compared to that of Ying. Also, while the asymptotic variance of maximum likelihood does not depend on the triangular array of observation points under consideration, that of cross v…
We consider general high-dimensional spiked sample covariance models and show that their leading sample spiked eigenvalues and their linear spectral statistics are asymptotically independent when the sample size and dimension are proportional to each other. As a byproduct, we also establish the central limit theorem of the leading sample spiked eigenvalues by removing the block diagonal assumption on the population covariance matrix, which is commonly needed in the literature. Moreover, we propose consistent estimators of the $L_4$ norm of the spiked population eigenvectors. Based on these res…
These expository notes serve as a reference for an accompanying post Morales-Jimenez et al. [2018]. In the spiked covariance model, we develop results on asymptotic normality of sample leading eigenvalues and certain projections of the corresponding sample eigenvectors. The results parallel those of Paul [2007], but are given using the non-Gaussian model of Bai and Yao [2008]. The results are not new, and citations are given, but proofs are collected and organized as a point of departure for Morales-Jimenez et al. [2018].
A common feature of high-dimensional data is that the data dimension is high, however, the sample size is relatively low. We call such data HDLSS data. In this paper, we study asymptotic properties of the first principal component in the HDLSS context and apply them to equality tests of covariance matrices for high dimensional data sets. We consider HDLSS asymptotic theories as the dimension grows for both the cases when the sample size is fixed and the sample size goes to infinity. We introduce an eigenvalue estimator by the noise-reduction methodology and provide asymptotic distributions of …
The association between two random variables is often of primary interest in statistical research. In this paper semiparametric models for the association between random vectors X and Y are considered which leave the marginal distributions arbitrary. Given that the odds ratio function comprises the whole information about the association the focus is on bilinear log-odds ratio models and in particular on the odds ratio parameter vector θ. The covariance structure of the maximum likelihood estimator θ^ of θ is of major importance for asymptotic inference. To this end different representations o…
There has been a growing interest in providing models for multivariate spatial processes. A majority of these models specify a parametric matrix covariance function. Based on observations, the parameters are estimated by maximum likelihood or variants thereof. While the asymptotic properties of maximum likelihood estimators for univariate spatial processes have been analyzed in detail, maximum likelihood estimators for multivariate spatial processes have not received their deserved attention yet. In this article we consider the classical increasing-domain asymptotic setting restricting the min…
The consistency and asymptotic normality of the spatial sign covariance matrix with unknown location are shown. Simulations illustrate the different asymptotic behavior when using the mean and the spatial median as location estimator.
We give two asymptotic results for the empirical distance covariance on separable metric spaces without any iid assumption on the samples. In particular, we show the almost sure convergence of the empirical distance covariance for any measure with finite first moments, provided that the samples form a strictly stationary and ergodic process. We further give a result concerning the asymptotic distribution of the empirical distance covariance under the assumption of absolute regularity of the samples and extend these results to certain types of pseudometric spaces. In the process, we derive a ge…
The asymptotic normality for a large family of eigenvalue statistics of a general sample covariance matrix is derived under the ultra-high dimensional setting, that is, when the dimension to sample size ratio $p/n \to \infty$. Based on this CLT result, we first adapt the covariance matrix test problem to the new ultra-high dimensional context. Then as a second application, we develop a new test for the separable covariance structure of a matrix-valued white noise. Simulation experiments are conducted for the investigation of finite-sample properties of the general asymptotic normality of eigen…
High dimensionality comparable to sample size is common in many statistical problems. We examine covariance matrix estimation in the asymptotic framework that the dimensionality $p$ tends to $\infty$ as the sample size $n$ increases. Motivated by the Arbitrage Pricing Theory in finance, a multi-factor model is employed to reduce dimensionality and to estimate the covariance matrix. The factors are observable and the number of factors $K$ is allowed to grow with $p$. We investigate impact of $p$ and $K$ on the performance of the model-based covariance matrix estimator. Under mild assumptions, w…
Assume that X is a set of sample statistics which follow a special case Central Limit Theorem, namely: as the sample size n increases the corresponding distribution becomes multivariate Normal with the mean (of each X) equal to zero and with an idempotent variance-covariance matrix V. It is well known that X^{T}X has (in the same limit), a chi-squared distribution with degrees of freedom equal to the trace of V. In this article we extend the above result to include the corresponding (1/n)-proportional corrections, making the new approximation substantially more accurate and extending its range…
Let $X,X_1,\dots, X_n$ be i.i.d. Gaussian random variables with zero mean and covariance operator $Σ={\mathbb E}(X\otimes X)$ taking values in a separable Hilbert space ${\mathbb H}.$ Let $$ {\bf r}(Σ):=\frac{{\rm tr}(Σ)}{\|Σ\|_{\infty}} $$ be the effective rank of $Σ,$ ${\rm tr}(Σ)$ being the trace of $Σ$ and $\|Σ\|_{\infty}$ being its operator norm. Let $$\hat Σ_n:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n).$ The paper deals with a problem of estimation of spectral projectors of the covariance operator $Σ$ by their empirical…
This paper is concerned with the problem of comparing the population means of two groups of independent observations. An approximate randomization test procedure based on the test statistic of Chen and Qin (2010) is proposed. The asymptotic behavior of the test statistic as well as the randomized statistic is studied under weak conditions. In our theoretical framework, observations are not assumed to be identically distributed even within groups. No condition on the eigenstructure of the covariance matrices is imposed. And the sample sizes of the two groups are allowed to be unbalanced. Under …
The goal of this study was to test the equality of two covariance matrices by using modified Pillai's trace statistics under a high-dimensional framework, i.e., the dimension and sample sizes go to infinity proportionally. In this paper, we introduce two modified Pillai's trace statistics and obtain their asymptotic distributions under the null hypothesis. The benefits of the proposed statistics include the following: (1) the sample size can be smaller than the dimensions; (2) the limiting distributions of the proposed statistics are universal; and (3) we do not restrict the structure of the p…
In many situations, when dealing with several populations, equality of the covariance operators is assumed. An important issue is to study if this assumption holds before making other inferences. In this paper, we develop a test for comparing covariance operators of several functional data samples. The proposed test is based on the Hilbert--Schmidt norm of the difference between estimated covariance operators. In particular, when dealing with two populations, the tests statistic is just the squared norm of the difference between the two covariance operators estimators. The asymptotic behaviour…
We study the asymptotic distributions of the spiked eigenvalues and the largest nonspiked eigenvalue of the sample covariance matrix under a general covariance matrix model with divergent spiked eigenvalues, while the other eigenvalues are bounded but otherwise arbitrary. The limiting normal distribution for the spiked sample eigenvalues is established. It has distinct features that the asymptotic mean relies on not only the population spikes but also the nonspikes and that the asymptotic variance in general depends on the population eigenvectors. In addition, the limiting Tracy-Widom law for …
We provide a unified approach to a method of estimation of the regression parameter in balanced linear models with a structured covariance matrix that combines a high breakdown point and bounded influence with high asymptotic efficiency at models with multivariate normal errors. Of main interest are linear mixed effects models, but our approach also includes several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of the estimators and corresponding functionals, es…
A joint limit theorem for the point process of the off-diagonal entries of a sample covariance matrix $\mathbf{S}$, constructed from $n$ observations of a $p$-dimensional random vector with iid components, and the Frobenius norm of $\mathbf{S}$ is proved. In particular, assuming that $p$ and $n$ tend to infinity we obtain a central limit theorem for the Frobenius norm in the case of finite fourth moment of the components and an infinite variance stable law in the case of infinite fourth moment. Extending a theorem of Kallenberg, we establish asymptotic independence of the point process and the…
We study parameter estimation in linear Gaussian covariance models, which are $p$-dimensional Gaussian models with linear constraints on the covariance matrix. Maximum likelihood estimation for this class of models leads to a non-convex optimization problem which typically has many local maxima. Using recent results on the asymptotic distribution of extreme eigenvalues of the Wishart distribution, we provide sufficient conditions for any hill-climbing method to converge to the global maximum. Although we are primarily interested in the case in which $n>\!\!>p$, the proofs of our results utiliz…
When the spatial sample size is extremely large, which occurs in many environmental and ecological studies, operations on the large covariance matrix are a numerical challenge. Covariance tapering is a technique to alleviate the numerical challenges. Under the assumption that data are collected along a line in a bounded region, we investigate how the tapering affects the asymptotic efficiency of the maximum likelihood estimator (MLE) for the microergodic parameter in the Matérn covariance function by establishing the fixed-domain asymptotic distribution of the exact MLE and that of the tapered…
This article presents a homogeneity test for testing the equality of several high-dimensional covariance matrices for stationary processes with ignoring the assumption of normality. We give the asymptotic distribution of the proposed test. The simulation illustrates that the proposed test has perfect performance. Moreover, the power of the test can approach any high probability uniformly on a set of covariance matrices.
We propose a robust test for the equality of the covariance structures in two functional samples. The test statistic has a chi-square asymptotic distribution with a known number of degrees of freedom, which depends on the level of dimension reduction needed to represent the data. Detailed analysis of the asymptotic properties is developed. Finite sample performance is examined by a simulation study and an application to egg-laying curves of fruit flies.
A multivariable measurement error model $AX \approx B$ is considered. Here $A$ and $B$ are input and output matrices of measurements and $X$ is a rectangular matrix of fixed size to be estimated. The errors in $[A,B]$ are row-wise independent, but within each row the errors may be correlated. Some of the columns are observed without errors and the error covariance matrices may differ from row to row. The total covariance structure of the errors is known up to a scalar factor. The fully weighted total least squares estimator of $X$ is studied. We give conditions for asymptotic normality of the …
Midzuno sampling enables to estimate ratios unbiasedly. We prove the asymptotic normality for estimators of totals and ratios under Midzuno sampling. We also propose consistent variance estimators.
We observe a sample of $n$ independent $p$-dimensional Gaussian vectors with Toeplitz covariance matrix $ Σ= [σ_{|i-j|}]_{1 \leq i,j \leq p}$ and $σ_0=1$. We consider the problem of testing the hypothesis that $Σ$ is the identity matrix asymptotically when $n \to \infty$ and $p \to \infty$. We suppose that the covariances $σ_k$ decrease either polynomially ($\sum_{k \geq 1} k^{2α} σ^2_{k} \leq L$ for $ α>1/4$ and $L>0$) or exponentially ($\sum_{k \geq 1} e^{2Ak} σ^2_{k} \leq L$ for $ A,L>0$). We consider a test procedure based on a weighted U-statistic of order 2, with optimal weights chosen a…
For a sample of $n$ independent identically distributed $p$-dimensional centered random vectors with covariance matrix $\mathbfΣ_n$ let $\tilde{\mathbf{S}}_n$ denote the usual sample covariance (centered by the mean) and $\mathbf{S}_n$ the non-centered sample covariance matrix (i.e. the matrix of second moment estimates), where $p> n$. In this paper, we provide the limiting spectral distribution and central limit theorem for linear spectral statistics of the Moore-Penrose inverse of $\mathbf{S}_n$ and $\tilde{\mathbf{S}}_n$. We consider the large dimensional asymptotics when the number of vari…
Under the high-dimensional setting that data dimension and sample size tend to infinity proportionally, we derive the central limit theorem (CLT) for linear spectral statistics (LSS) of large-dimensional sample covariance matrix. Different from existing literature, our results do not require the assumption that the population covariance matrices are bounded. Moreover, many common kernel functions in the real data such as logarithmic functions and polynomial functions are allowed in this paper. In our model, the number of spiked eigenvalues can be fixed or tend to infinity. One salient feature …
Consider $n$ independent and identically distributed $p$-dimensional Gaussian random vectors with covariance matrix $Σ.$ The problem of estimating $Σ$ when $p$ is much larger than $n$ has received a lot of attention in recent years. Yet little is known about the information criterion for covariance matrix estimation. How to properly define such a criterion and what are the statistical properties? We attempt to answer these questions in the present paper by focusing on the estimation of bandable covariance matrices when $p>n$ but $\log(p)=o(n)$. Motivated by the deep connection between Stein's …
We study the distribution of the maximum likelihood estimate (MLE) in high-dimensional logistic models, extending the recent results from Sur (2019) to the case where the Gaussian covariates may have an arbitrary covariance structure. We prove that in the limit of large problems holding the ratio between the number $p$ of covariates and the sample size $n$ constant, every finite list of MLE coordinates follows a multivariate normal distribution. Concretely, the $j$th coordinate $\hat β_j$ of the MLE is asymptotically normally distributed with mean $α_\star β_j$ and standard deviation $σ_\star/…
When shrinking a covariance matrix towards (a multiple) of the identity matrix, the trace of the covariance matrix arises naturally as the optimal scaling factor for the identity target. The trace also appears in other context, for example when measuring the size of a matrix or the amount of uncertainty. Of particular interest is the case when the dimension of the covariance matrix is large. Then the problem arises that the sample covariance matrix is singular if the dimension is larger than the sample size. Another issue is that usually the estimation has to based on correlated time series da…
The assumption of separability of the covariance operator for a random image or hypersurface can be of substantial use in applications, especially in situations where the accurate estimation of the full covariance structure is unfeasible, either for computational reasons, or due to a small sample size. However, inferential tools to verify this assumption are somewhat lacking in high-dimensional or functional {data analysis} settings, where this assumption is most relevant. We propose here to test separability by focusing on $K$-dimensional projections of the difference between the covariance o…
In this paper, we propose two new tests for testing the equality of the covariance functions of several functional populations, namely a quasi GPF test and a quasi $F_{\max}$ test. The asymptotic random expressions of the two tests under the null hypothesis are derived. We show that the asymptotic null distribution of the quasi GPF test is a chi-squared-type mixture whose distribution can be well approximated by a simple scaled chi-squared distribution. We also adopt a random permutation method for approximating the null distributions of the quasi GPF and $F_{\max}$ tests. The random permutati…
In many statistical signal processing applications, the estimation of nuisance parameters and parameters of interest is strongly linked to the resulting performance. Generally, these applications deal with complex data. This paper focuses on covariance matrix estimation problems in non-Gaussian environments and particularly, the M-estimators in the context of elliptical distributions. Firstly, this paper extends to the complex case the results of Tyler in [1]. More precisely, the asymptotic distribution of these estimators as well as the asymptotic distribution of any homogeneous function of d…
The Matérn covariance function is a popular choice for prediction in spatial statistics and uncertainty quantification literature. A key benefit of the Matérn class is that it is possible to get precise control over the degree of mean-square differentiability of the random process. However, the Matérn class possesses exponentially decaying tails, and thus may not be suitable for modeling polynomially decaying dependence. This problem can be remedied using polynomial covariances; however one loses control over the degree of mean-square differentiability of corresponding processes, in that rando…
We undertake a precise study of the asymptotic and non-asymptotic properties of stochastic approximation procedures with Polyak-Ruppert averaging for solving a linear system $\bar{A} θ= \bar{b}$. When the matrix $\bar{A}$ is Hurwitz, we prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity. The CLT characterizes the exact asymptotic covariance matrix, which is the sum of the classical Polyak-Ruppert covariance and a correction term that scales with the step size. Under assumptions on the tail of the noise distribution, we …
Parsimonious Compactly Supported Covariance Models in the Gauss Hypergeometric Class: Identifiability, Reparameterizations, and Asymptotic Properties
Universidad Adolfo Ibáñez（阿道夫·伊巴涅斯大学）；Università Ca’ Foscari（威尼斯卡福斯卡里大学）；Universidad del Bío-Bío（比奥比奥大学）；Universidad de Santiago de Chile（智利圣地亚哥大学）；Universidad de Chile（智利大学）
The main theme of this paper is a modification of the likelihood ratio test (LRT) for testing high dimensional covariance matrix. Recently, the correct asymptotic distribution of the LRT for a large-dimensional case (the case $p/n$ approaches to a constant $γ\in (0,1]$) is specified by researchers. The correct procedure is named as corrected LRT. Despite of its correction, the corrected LRT is a function of sample eigenvalues that are suffered from redundant variability from high dimensionality and, subsequently, still does not have full power in differentiating hypotheses on the covariance ma…
Estimation and hypothesis tests for the covariance matrix in high dimensions is a challenging problem as the traditional multivariate asymptotic theory is no longer valid. When the dimension is larger than or increasing with the sample size, standard likelihood based tests for the covariance matrix have poor performance. Existing high dimensional tests are either computationally expensive or have very weak control of type I error. In this paper, we propose a test procedure, CRAMP, for testing hypotheses involving one or more covariance matrices using random projections. Projecting the high dim…
