This article presents a homogeneity test for testing the equality of several high-dimensional covariance matrices for stationary processes with ignoring the assumption of normality. We give the asymptotic distribution of the proposed test. The simulation illustrates that the proposed test has perfect performance. Moreover, the power of the test can approach any high probability uniformly on a set of covariance matrices.
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
The asymptotic normality for a large family of eigenvalue statistics of a general sample covariance matrix is derived under the ultra-high dimensional setting, that is, when the dimension to sample size ratio $p/n \to \infty$. Based on this CLT result, we first adapt the covariance matrix test problem to the new ultra-high dimensional context. Then as a second application, we develop a new test for the separable covariance structure of a matrix-valued white noise. Simulation experiments are conducted for the investigation of finite-sample properties of the general asymptotic normality of eigen…
We consider estimation of the covariance matrix of a multivariate random vector under the constraint that certain covariances are zero. We first present an algorithm, which we call Iterative Conditional Fitting, for computing the maximum likelihood estimator of the constrained covariance matrix, under the assumption of multivariate normality. In contrast to previous approaches, this algorithm has guaranteed convergence properties. Dropping the assumption of multivariate normality, we show how to estimate the covariance matrix in an empirical likelihood approach. These approaches are then compa…
A multivariable measurement error model $AX \approx B$ is considered. Here $A$ and $B$ are input and output matrices of measurements and $X$ is a rectangular matrix of fixed size to be estimated. The errors in $[A,B]$ are row-wise independent, but within each row the errors may be correlated. Some of the columns are observed without errors and the error covariance matrices may differ from row to row. The total covariance structure of the errors is known up to a scalar factor. The fully weighted total least squares estimator of $X$ is studied. We give conditions for asymptotic normality of the …
This article provides an introduction to the asymptotic analysis of covariance parameter estimation for Gaussian processes. Maximum likelihood estimation is considered. The aim of this introduction is to be accessible to a wide audience and to present some existing results and proof techniques from the literature. The increasing-domain and fixed-domain asymptotic settings are considered. Under increasing-domain asymptotics, it is shown that in general all the components of the covariance parameter can be estimated consistently by maximum likelihood and that asymptotic normality holds. In contr…
The consistency and asymptotic normality of the spatial sign covariance matrix with unknown location are shown. Simulations illustrate the different asymptotic behavior when using the mean and the spatial median as location estimator.
We propose an independence test for random variables valued into metric spaces by using a test statistic obtained from appropriately centering and rescaling the squared Hilbert-Schmidt norm of the usual empirical estimator of normalized cross-covariance operator. We then get asymptotic normality of this statistic under independence hypothesis, so leading to a new test for independence of functional random variables. A simulation study that allows to compare the proposed test to existing ones is provided.
In this paper, the uniformly asymptotic normality for sample quantiles of associated random variables is investigated under some conditions on the decay of the covariances. We obtain the rate of normal approximation of order $O(n^{-1/2}\log^2 n)$ if the covariances decrease exponentially to $0$. The best rate is shown as $O(n^{-1/3})$ under a polynomial decay of the covariances.
Asymptotic distribution for the proportional covariance model under multivariate normal distributions is derived. To this end, the parametrization of the common covariance matrix by its Cholesky root is adopted. The derivations are made in three steps. First, the asymptotic distribution of the maximum likelihood estimators of the proportionality coefficients and the Cholesky inverse root of the common covariance matrix is derived by finding the information matrix and its inverse. Next, the asymptotic distributions for the case of the Cholesky root of the common covariance matrix and finally fo…
We consider inference problems for high-dimensional (HD) functional data with a dense number (T) of repeated measurements taken for a large number of p variables from a small number of n experimental units. The spatial and temporal dependence, high dimensionality, and the dense number of repeated measurements all make theoretical studies and computation challenging. This paper has two aims; our first aim is to solve the theoretical and computational challenges in detecting and identifying change points among covariance matrices from HD functional data. The second aim is to provide computationa…
These expository notes serve as a reference for an accompanying post Morales-Jimenez et al. [2018]. In the spiked covariance model, we develop results on asymptotic normality of sample leading eigenvalues and certain projections of the corresponding sample eigenvectors. The results parallel those of Paul [2007], but are given using the non-Gaussian model of Bai and Yao [2008]. The results are not new, and citations are given, but proofs are collected and organized as a point of departure for Morales-Jimenez et al. [2018].
Let $X,X_1,\dots, X_n$ be i.i.d. Gaussian random variables with zero mean and covariance operator $Σ={\mathbb E}(X\otimes X)$ taking values in a separable Hilbert space ${\mathbb H}.$ Let $$ {\bf r}(Σ):=\frac{{\rm tr}(Σ)}{\|Σ\|_{\infty}} $$ be the effective rank of $Σ,$ ${\rm tr}(Σ)$ being the trace of $Σ$ and $\|Σ\|_{\infty}$ being its operator norm. Let $$\hat Σ_n:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n).$ The paper deals with a problem of estimation of spectral projectors of the covariance operator $Σ$ by their empirical…
We observe a sample of $n$ independent $p$-dimensional Gaussian vectors with Toeplitz covariance matrix $ Σ= [σ_{|i-j|}]_{1 \leq i,j \leq p}$ and $σ_0=1$. We consider the problem of testing the hypothesis that $Σ$ is the identity matrix asymptotically when $n \to \infty$ and $p \to \infty$. We suppose that the covariances $σ_k$ decrease either polynomially ($\sum_{k \geq 1} k^{2α} σ^2_{k} \leq L$ for $ α>1/4$ and $L>0$) or exponentially ($\sum_{k \geq 1} e^{2Ak} σ^2_{k} \leq L$ for $ A,L>0$). We consider a test procedure based on a weighted U-statistic of order 2, with optimal weights chosen a…
The author uses a Stein-type covariance identity to obtain moment estimators for the parameters of the quadratic polynomial subfamily of Pearson distributions. The asymptotic distribution of the estimators is obtained, and normality and symmetry tests based on it are provided. Simulation is used to compare the performance of the proposed tests with that of other existing tests for symmetry and normality.
In arXiv:0907.0079 by Cator and Lopuhaa, an asymptotic expansion for the MCD estimators is established in a very general framework. This expansion requires the existence and non-singularity of the derivative in a first-order Taylor expansion. In this paper, we prove the existence of this derivative for multivariate distributions that have a density and provide an explicit expression. Moreover, under suitable symmetry conditions on the density, we show that this derivative is non-singular. These symmetry conditions include the elliptically contoured multivariate location-scatter model, in which…
High dimensionality comparable to sample size is common in many statistical problems. We examine covariance matrix estimation in the asymptotic framework that the dimensionality $p$ tends to $\infty$ as the sample size $n$ increases. Motivated by the Arbitrage Pricing Theory in finance, a multi-factor model is employed to reduce dimensionality and to estimate the covariance matrix. The factors are observable and the number of factors $K$ is allowed to grow with $p$. We investigate impact of $p$ and $K$ on the performance of the model-based covariance matrix estimator. Under mild assumptions, w…
In this paper, we consider an inference problem for the first order autoregressive process with non-zero mean driven by a long memory stationary Gaussian process. Suppose that the covariance function of the noise can be expressed as $|k|^{2H-2}$ times a positive constant when $k$ tends to infinity, and the fractional Gaussian noise and the fractional ARIMA model are special examples that satisfy this assumption. We propose moment estimators and prove the strong consistency, the asymptotic normality and joint asymptotic normality.
We consider settings where the observations are drawn from a zero-mean multivariate (real or complex) normal distribution with the population covariance matrix having eigenvalues of arbitrary multiplicity. We assume that the eigenvectors of the population covariance matrix are unknown and focus on inferential procedures that are based on the sample eigenvalues alone (i.e., "eigen-inference"). Results found in the literature establish the asymptotic normality of the fluctuation in the trace of powers of the sample covariance matrix. We develop concrete algorithms for analytically computing the …
This paper explores the homogeneity of coefficients in high-dimensional regression, which extends the sparsity concept and is more general and suitable for many applications. Homogeneity arises when one expects regression coefficients corresponding to neighboring geographical regions or a similar cluster of covariates to be approximately the same. Sparsity corresponds to a special case of homogeneity with a known atom zero. In this article, we propose a new method called clustering algorithm in regression via data-driven segmentation (CARDS) to explore homogeneity. New mathematics are provided…
In this work, we are motivated by the recent work of Zhang et al. (2019) and study a new invariant test for equality of two large scale covariance matrices. Two modified likelihood ratio tests (LRTs) by Zhang et al. (2019) are based on the sum of log of eigenvalues (or 1- eigenvalues) of the Beta-matrix. However, as the dimension increases, many eigenvalues of the Beta-matrix are close to 0 or 1 and the modified LRTs are greatly influenced by them. In this work, instead, we consider the simple sum of the eigenvalues (of the Beta-matrix) and compute its asymptotic normality when all $n_1, n_2, …
The present manuscript studies signal detection by likelihood ratio tests in a number of spiked random matrix models, including but not limited to Gaussian mixtures and spiked Wishart covariance matrices. We work directly with multi-spiked cases in these models and with flexible priors on the signal component that allow dependence across spikes. We derive asymptotic normality for the log-likelihood ratios when the signal-to- noise ratios are below certain thresholds. In addition, we show that the variances of the log-likelihood ratios can be asymptotically decomposed as the sums of those of a …
This paper deals with the problem of identifying and estimating dynamical parameters of continuous-time quantum open systems, in the input-output formalism. First, we characterise the space of identifiable parameters for ergodic dynamics, assuming full access to the output state for arbitrarily long times, and show that the equivalence classes of undistinguishable parameters are orbits of a Lie group acting on the space of dynamical parameters. Second, we define an information geometric structure on this space, including a principal bundle given by the action of the group, as well as a compati…
We consider a one-dimensional Gaussian process having exponential covariance function. Under fixed-domain asymptotics, we prove the strong consistency and asymptotic normality of a cross validation estimator of the microergodic covariance parameter. In this setting, Ying [40] proved the same asymptotic properties for the maximum likelihood estimator. Our proof includes several original or more involved components, compared to that of Ying. Also, while the asymptotic variance of maximum likelihood does not depend on the triangular array of observation points under consideration, that of cross v…
In this paper we study the asymptotic normality in high-dimensional linear regression. We focus on the case where the covariance matrix of the regression variables has a KMS structure, in asymptotic settings where the number of predictors, $p$, is proportional to the number of observations, $n$. The main result of the paper is the derivation of the exact asymptotic distribution for the suitably centered and normalized squared norm of the product between predictor matrix, $\mathbb{X}$, and outcome variable, $Y$, i.e. the statistic $\|\mathbb{X}'Y\|_{2}^{2}$. Additionally, we consider a specific…
We establish bounds for the covariance of a large class of functions of infinite variance stable random variables, including unbounded functions such as the power function and the logarithm. These bounds involve measures of dependence between the stable variables, some of which are new. The bounds are also used to deduce the central limit theorem for unbounded functions of stable moving average time series. This result extends the earlier results of Tailen Hsing and the authors on central limit theorems for bounded functions of stable moving averages. It can be used to show asymptotic normalit…
We consider the problem of selecting covariates in spatial linear models with Gaussian process errors. Penalized maximum likelihood estimation (PMLE) that enables simultaneous variable selection and parameter estimation is developed and, for ease of computation, PMLE is approximated by one-step sparse estimation (OSE). To further improve computational efficiency, particularly with large sample sizes, we propose penalized maximum covariance-tapered likelihood estimation (PMLE$_{\mathrm{T}}$) and its one-step sparse estimation (OSE$_{\mathrm{T}}$). General forms of penalty functions with an emph…
In their seminal work, Polyak and Juditsky showed that stochastic approximation algorithms for solving smooth equations enjoy a central limit theorem. Moreover, it has since been argued that the asymptotic covariance of the method is best possible among any estimation procedure in a local minimax sense of Hájek and Le Cam. A long-standing open question in this line of work is whether similar guarantees hold for important non-smooth problems, such as stochastic nonlinear programming or stochastic variational inequalities. In this work, we show that this is indeed the case.
We provide a unified approach to a method of estimation of the regression parameter in balanced linear models with a structured covariance matrix that combines a high breakdown point and bounded influence with high asymptotic efficiency at models with multivariate normal errors. Of main interest are linear mixed effects models, but our approach also includes several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of the estimators and corresponding functionals, es…
We provide a unified approach to S-estimation in balanced linear models with structured covariance matrices. Of main interest are S-estimators for linear mixed effects models, but our approach also includes S-estimators in several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of S-functionals and S-estimators, establish asymptotic properties such as consistency and asymptotic normality, and derive their robustness properties in terms of breakdown point and influ…
We define the minimum covariance determinant functionals for multivariate location and scatter through trimming functions and establish their existence at any multivariate distribution. We provide a precise characterization including a separating ellipsoid property and prove that the functionals are continuous. Moreover, we establish asymptotic normality for both the location and covariance estimator and derive the influence function. These results are obtained in a very general multivariate setting.
Let $\mathbf{Y}=\mathbf{X}\boldsΘ\mathbf{Z}'+\bolds{\mathcal {E}}$ be the growth curve model with $\bolds{\mathcal{E}}$ distributed with mean $\mathbf{0}$ and covariance $\mathbf{I}_n\otimes\boldsΣ$, where $\boldsΘ$, $\boldsΣ$ are unknown matrices of parameters and $\mathbf{X}$, $\mathbf{Z}$ are known matrices. For the estimable parametric transformation of the form $\bolds γ=\mathbf{C}\boldsΘ\mathbf{D}'$ with given $\mathbf{C}$ and $\mathbf{D}$, the two-stage generalized least-squares estimator $\hat{\bolds γ}(\mathbf{Y})$ defined in (7) converges in probability to $\boldsγ$ as the sample siz…
This paper considers the statistical inference of the class of asymmetric power-transformed $\operatorname{GARCH}(1,1)$ models in presence of possible explosiveness. We study the explosive behavior of volatility when the strict stationarity condition is not met. This allows us to establish the asymptotic normality of the quasi-maximum likelihood estimator (QMLE) of the parameter, including the power but without the intercept, when strict stationarity does not hold. Two important issues can be tested in this framework: asymmetry and stationarity. The tests exploit the existence of a universal e…
Longitudinal studies are often conducted to explore the cohort and age effects in many scientific areas. The within cluster correlation structure plays a very important role in longitudinal data analysis. This is because not only can an estimator be improved by incorporating the within cluster correlation structure into the estimation procedure, but also the within cluster correlation structure can sometimes provide valuable insights in practical problems. For example, it can reveal the correlation strengths among the impacts of various factors. Motivated by data typified by a set from Banglad…
The assumption of separability is a simplifying and very popular assumption in the analysis of spatio-temporal or hypersurface data structures. It is often made in situations where the covariance structure cannot be easily estimated, for example because of a small sample size or because of computational storage problems. In this paper we propose a new and very simple test to validate this assumption. Our approach is based on a measure of separability which is zero in the case of separability and positive otherwise. The measure can be estimated without calculating the full non-separable covaria…
Online Covariance Matrix Estimation in Sketched Newton Methods
Department of Statistics, The University of Chicago（芝加哥大学统计系）；Mathematics and Computer Science Division, Argonne National Laboratory（阿贡国家实验室数学与计算机科学部）；School of Industrial and Systems Engineering, Georgia Institute of Technology（佐治亚理工学院工业与系统工程系）
We study the distributional properties of the linear discriminant function under the assumption of normality by comparing two groups with the same covariance matrix but different mean vectors. A stochastic representation for the discriminant function coefficients is derived which is then used to obtain their asymptotic distribution under the high-dimensional asymptotic regime. We investigate the performance of the classification analysis based on the discriminant function in both small and large dimensions. A stochastic representation is established which allows to compute the error rate in an…
This manuscript makes two contributions to the field of change-point detection. In a generalchange-point setting, we provide a generic algorithm for aggregating local homogeneity testsinto an estimator of change-points in a time series. Interestingly, we establish that the errorrates of the collection of tests directly translate into detection properties of the change-pointestimator. This generic scheme is then applied to various problems including covariance change-point detection, nonparametric change-point detection and sparse multivariate mean change-point detection. For the latter, we der…
The matrix-variate normal distribution is a popular model for high-dimensional transposable data because it decomposes the dependence structure of the random matrix into the Kronecker product of two covariance matrices: one for each of the row and column variables. We develop tests for assessing the form of the row (column) covariance matrix in high-dimensional settings while treating the column (row) dependence structure as a nuisance. Our tests are robust to normality departures provided that the Kronecker product dependence structure holds. In simulations, we observe that the proposed tests…
