An admissible estimator of the eigenvalues of the variance-covariance matrix is given for multivariate normal distributions with respect to the scale-invariant squared error loss.
We study admissibility of a subclass of generalized Bayes estimators of a multivariate normal vector when the variance is unknown, under scaled quadratic loss. Minimaxity is also established for certain of these estimators.
So far, the pseudo cross-variogram is primarily used as a tool for the structural analysis of multivariate random fields. Mainly applying recent theoretical results on the pseudo cross-variogram, we use it as a cornerstone in the construction of valid covariance models for multivariate random fields. In particular, we extend known univariate constructions to the multivariate case, and generalize existing multivariate models. Furthermore, we provide a general construction principle for conditionally negative definite matrix-valued kernels, which we use to reinterpret previous modeling proposals…
Matrix-valued covariance functions are crucial to geostatistical modeling of multivariate spatial data. The classical assumption of symmetry of a multivariate covariance function is overlay restrictive and has been considered as unrealistic for most of real data applications. Despite of that, the literature on asymmetric covariance functions has been very sparse. In particular, there is some work related to asymmetric covariances on Euclidean spaces, depending on the Euclidean distance. However, for data collected over large portions of planet Earth, the most natural spatial domain is a sphere…
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
There has been a growing interest in providing models for multivariate spatial processes. A majority of these models specify a parametric matrix covariance function. Based on observations, the parameters are estimated by maximum likelihood or variants thereof. While the asymptotic properties of maximum likelihood estimators for univariate spatial processes have been analyzed in detail, maximum likelihood estimators for multivariate spatial processes have not received their deserved attention yet. In this article we consider the classical increasing-domain asymptotic setting restricting the min…
We consider an inference on the eigenvalues of the covariance matrix of a multivariate normal distribution. The family of multivariate normal distributions with a fixed mean is seen as a Riemannian manifold with Fisher information metric. Two submanifolds naturally arises; one is the submanifold given by fixed eigenvectors of the covariance matrix, the other is the one given by fixed eigenvalues. We analyze the geometrical structures of these manifolds such as metric, embedding curvature under $e$-connection or $m$-connection. Based on these results, we study 1) the bias of the sample eigenval…
The main objective of this work is to calculate the multivariate double truncated expectation (MDTE) and covariance (MDTCov) for elliptical distributions. We also consider double truncated expectation (DTE) and variance (DTV) for univariate elliptical distributions. The exact expressions of MDTE and MDTCov are derived for some special cases of the family, such as normal, student-$t$, logistic, Laplace and Pearson type VII distributions. As numerical illustration, the DTE, DTV, MDTE and MDTCov for normal distribution are computed in details. Finally, we discuss MDTE and MDTCov of three industry…
The construction of valid and flexible cross-covariance functions is a fundamental task for modeling multivariate space-time data arising from climatological and oceanographical phenomena. Indeed, a suitable specification of the covariance structure allows to capture both the space-time dependencies between the observations and the development of accurate predictions. For data observed over large portions of planet Earth it is necessary to take into account the curvature of the planet. Hence the need for random field models defined over spheres across time. In particular, the associated covari…
In this paper, the multivariate tail covariance (MTCov) for generalized skew-elliptical distributions is considered. Some special cases for this distribution, such as generalized skew-normal, generalized skew student-t, generalized skew-logistic and generalized skew-Laplace distributions, are also considered. In order to test the theoretical feasibility of our results, the MTCov for skewed and non skewed normal distributions are computed and compared. Finally, we give a special formula of the MTCov for generalized skew-elliptical distributions.
We provide a unified approach to a method of estimation of the regression parameter in balanced linear models with a structured covariance matrix that combines a high breakdown point and bounded influence with high asymptotic efficiency at models with multivariate normal errors. Of main interest are linear mixed effects models, but our approach also includes several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of the estimators and corresponding functionals, es…
For multivariate stationary time series many important properties, such as partial correlation, graphical models and autoregressive representations are encoded in the inverse of its spectral density matrix. This is not true for nonstationary time series, where the pertinent information lies in the inverse infinite dimensional covariance matrix operator associated with the multivariate time series. This necessitates the study of the covariance of a multivariate nonstationary time series and its relationship to its inverse. We show that if the rows/columns of the infinite dimensional covariance …
In this paper, we propose a novel variable selection approach in the framework of multivariate linear models taking into account the dependence that may exist between the responses. It consists in estimating beforehand the covariance matrix of the responses and to plug this estimator in a Lasso criterion, in order to obtain a sparse estimator of the coefficient matrix. The properties of our approach are investigated both from a theoretical and a numerical point of view. More precisely, we give general conditions that the estimators of the covariance matrix and its inverse have to satisfy in or…
Analyzing principal components for multivariate data from its spatial sign covariance matrix (SCM) has been proposed as a computationally simple and robust alternative to normal PCA, but it suffers from poor efficiency properties and is actually inadmissible with respect to the maximum likelihood estimator. Here we use data depth-based spatial ranks in place of spatial signs to obtain the orthogonally equivariant Depth Covariance Matrix (DCM) and use its eigenvector estimates for PCA. We derive asymptotic properties of the sample DCM and influence functions of its eigenvectors. The shapes of t…
This paper considers a multivariate spatial random field, with each component having univariate marginal distributions of the skew-Gaussian type. We assume that the field is defined spatially on the unit sphere embedded in $\mathbb{R}^3$, allowing for modeling data available over large portions of planet Earth. This model admits explicit expressions for the marginal and cross covariances. However, the $n$-dimensional distributions of the field are difficult to evaluate, because it requires the sum of $2^n$ terms involving the cumulative and probability density functions of a $n$-dimensional Ga…
This paper addresses the problem of finding parametric constraints that ensure the validity of the multivariate Mat{é}rn covariance for modeling the spatial correlation structure of coregionalized variables defined in an Euclidean space. To date, much attention has been given to the bivariate setting, while the multivariate setting has been explored to a limited extent only. The existing conditions often imply severe restrictions on the upper bounds for the collocated correlation coefficients, which makes the multivariate Mat{é}rn model appealing for the case of weak spatial cross-dependence o…
Monitoring several correlated quality characteristics of a process is common in modern manufacturing and service industries. Although a lot of attention has been paid to monitoring the multivariate process mean, not many control charts are available for monitoring the covariance matrix. This paper presents a comprehensive overview of the literature on control charts for monitoring the covariance matrix in a multivariate statistical process monitoring (MSPM) framework. It classifies the research that has previously appeared in the literature. We highlight the challenging areas for research and …
We provide a unified approach to S-estimation in balanced linear models with structured covariance matrices. Of main interest are S-estimators for linear mixed effects models, but our approach also includes S-estimators in several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of S-functionals and S-estimators, establish asymptotic properties such as consistency and asymptotic normality, and derive their robustness properties in terms of breakdown point and influ…
Asymptotic distribution for the proportional covariance model under multivariate normal distributions is derived. To this end, the parametrization of the common covariance matrix by its Cholesky root is adopted. The derivations are made in three steps. First, the asymptotic distribution of the maximum likelihood estimators of the proportionality coefficients and the Cholesky inverse root of the common covariance matrix is derived by finding the information matrix and its inverse. Next, the asymptotic distributions for the case of the Cholesky root of the common covariance matrix and finally fo…
The Behrens-Fisher problem is a well-known hypothesis testing problem in statistics concerning two-sample mean comparison. In this article, we confirm one conjecture in Eaton and Olshen (1972), which provides stochastic bounds for the multivariate Behrens-Fisher test statistic under the null hypothesis. We also extend their results on the stochastic ordering of random quotients to the arbitrary finite dimensional case. This work can also be seen as a generalization of Hsu (1938) that provided the bounds for the univariate Behrens-Fisher problem. The results obtained in this article can be used…
We consider the problem of estimating covariance and precision matrices, and their associated discriminant coefficients, from normal data when the rank of the covariance matrix is strictly smaller than its dimension and the available sample size. Using unbiased risk estimation, we construct novel estimators by minimizing upper bounds on the difference in risk over several classes. Our proposal estimates are empirically demonstrated to offer substantial improvement over classical approaches.
The paper deals with multivariate Gaussian random fields defined over generalized product spaces that involve the hypertorus. The assumption of Gaussianity implies the finite dimensional distributions to be completely specified by the covariance functions, being in this case matrix valued mappings. We start by considering the spectral representations that in turn allow for a characterization of such covariance functions. We then provide some methods for the construction of these matrix valued mappings. Finally, we consider strategies to evade radial symmetry (called isotropy in spatial statist…
The distance covariance of Székely, et al. [23] and Székely and Rizzo [21], a powerful measure of dependence between sets of multivariate random variables, has the crucial feature that it equals zero if and only if the sets are mutually independent. Hence the distance covariance can be applied to multivariate data to detect arbitrary types of non-linear associations between sets of variables. We provide in this article a basic, albeit rigorous, introductory treatment of the distance covariance. Our investigations yield an approach that can be used as the foundation for presentation of this imp…
In this paper, we focus on multivariate doubly truncated first two moments of generalized skew-elliptical (GSE) distributions and derive explicit expressions for them. It includes many useful distributions, for examples, generalized skew-normal (GSN), generalized skew-Laplace (GSLa), generalized skew-logistic (GSLo) and generalized skew student-$t$ (GSSt) distributions, all as special cases. We also give formulas of multivariate doubly truncated expectation and covariance for GSE distributions. As applications, we show the results of multivariate tail conditional expectation (MTCE) and multiva…
This paper is concerned with the small area estimation in the multivariate Fay-Herriot model where covariance matrix of random effects are fully unknown. The covariance matrix is estimated by a Prasad-Rao type consistent estimator, and the empirical best linear un- biased predictor (EBLUP) of a vector of small area characteristics is provided. When the EBLUP is measured in terms of a mean squared error matrix (MSEM), a second-order approximation of MSEM of the EBLUP and a second-order unbiased estimator of the MSEM is derived analytically in closed forms. The performance is investigated throug…
Multivariate time series present many challenges, especially when they are high dimensional. The paper's focus is twofold. First, we address the subject of consistently estimating the autocovariance sequence; this is a sequence of matrices that we conveniently stack into one huge matrix. We are then able to show consistency of an estimator based on the so-called flat-top tapers; most importantly, the consistency holds true even when the time series dimension is allowed to increase with the sample size. Second, we revisit the linear process bootstrap (LPB) procedure proposed by McMurry and Poli…
Multivariate processes with long-range dependent properties are found in a large number of applications including finance, geophysics and neuroscience. For real data applications, the correlation between time series is crucial. Usual estimations of correlation can be highly biased due to phase-shifts caused by the differences in the properties of autocorrelation in the processes. To address this issue, we introduce a semiparametric estimation of multivariate long-range dependent processes. The parameters of interest in the model are the vector of the long-range dependence parameters and the lo…
We consider high-dimensional multivariate linear regression models, where the joint distribution of covariates and response variables is a multivariate normal distribution with a bandable covariance matrix. The main goal of this paper is to estimate the regression coefficient matrix, which is a function of the bandable covariance matrix. Although the tapering estimator of covariance has the minimax optimal convergence rate for the class of bandable covariances, we show that it has a sub-optimal convergence rate for the regression coefficient; that is, a minimax estimator for the class of banda…
A multivariable measurement error model $AX \approx B$ is considered. Here $A$ and $B$ are input and output matrices of measurements and $X$ is a rectangular matrix of fixed size to be estimated. The errors in $[A,B]$ are row-wise independent, but within each row the errors may be correlated. Some of the columns are observed without errors and the error covariance matrices may differ from row to row. The total covariance structure of the errors is known up to a scalar factor. The fully weighted total least squares estimator of $X$ is studied. We give conditions for asymptotic normality of the …
We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance covariance. All the measures share similar properties and asymptotic distributions to distance covariance, and capture non-linear and non-monotone mutual dependence between the random vectors. Inspired by complete and incomplete V-statistics, we define the empirical measures and…
For a multivariate normal distribution, the sparsity of the covariance and precision matrices encodes complete information about independence and conditional independence properties. For general distributions, the covariance and precision matrices reveal correlations and so-called partial correlations between variables, but these do not, in general, have any correspondence with respect to independence properties. In this paper, we prove that, for a certain class of non-Gaussian distributions, these correspondences still hold, exactly for the covariance and approximately for the precision. The …
In recent years, shrinkage priors have received much attention in high-dimensional data analysis from a Bayesian perspective. Compared with widely used spike-and-slab priors, shrinkage priors have better computational efficiency. But the theoretical properties, especially posterior contraction rate, which is important in uncertainty quantification, are not established in many cases. In this paper, we apply global-local shrinkage priors to high-dimensional multivariate linear regression with unknown covariance matrix. We show that when the prior is highly concentrated near zero and has heavy ta…
University of Toronto(多伦多大学);Institute for Aerospace Studies(航空航天研究所)
This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there are some advantages to choosing a parameterization comprising the mean and inverse covariance matrix and provide a simple NGD update that accounts for the symmetric (and sparse) nature of the inverse covariance matrix.
The covariance structure of multivariate functional data can be highly complex, especially if the multivariate dimension is large, making extensions of statistical methods for standard multivariate data to the functional data setting challenging. For example, Gaussian graphical models have recently been extended to the setting of multivariate functional data by applying multivariate methods to the coefficients of truncated basis expansions. However, a key difficulty compared to multivariate data is that the covariance operator is compact, and thus not invertible. The methodology in this paper …
The need to test whether two random vectors are independent has spawned a large number of competing measures of dependence. We are interested in nonparametric measures that are invariant under strictly increasing transformations, such as Kendall's tau, Hoeffding's D, and the more recently discovered Bergsma--Dassios sign covariance. Each of these measures exhibits symmetries that are not readily apparent from their definitions. Making these symmetries explicit, we define a new class of multivariate nonparametric measures of dependence that we refer to as Symmetric Rank Covariances. This new cl…
In this paper, we address the problem of estimating a covariance matrix of a multivariate Gaussian distribution, relative to a Stein loss function, from a decision theoretic point of view. We investigate the case where the covariance matrix is invertible and the case when it is non--invertible in a unified approach.
The Fisher-Rao geodesic distance on the statistical manifold consisting of zero-mean p-dimensional multivariate Gaussians appears without proof in several places (such as Steven Smith's "Covariance, Subspace, and Intrinsic Cramer-Rao Bounds"). In this paper, we give a proof using basic Riemannian geometry.
We investigate predictive densities for multivariate normal models with unknown mean vectors and known covariance matrices. Bayesian predictive densities based on shrinkage priors often have complex representations, although they are effective in various problems. We consider extended normal models with mean vectors and covariance matrices as parameters, and adopt predictive densities that belong to the extended models including the original normal model. We adopt predictive densities that are optimal with respect to the posterior Bayes risk in the extended models. The proposed predictive dens…
This work aims to give non-asymptotic results for estimating the first principal component of a multivariate random process. We first define the covariance function and the covariance operator in the multivariate case. We then define a projection operator. This operator can be seen as a reconstruction step from the raw data in the functional data analysis context. Next, we show that the eigenelements can be expressed as the solution to an optimization problem, and we introduce the LASSO variant of this optimization problem and the associated plugin estimator. Finally, we assess the estimator's…
Many inference techniques for multivariate data analysis assume that the rows of the data matrix are realizations of independent and identically distributed random vectors. Such an assumption will be met, for example, if the rows of the data matrix are multivariate measurements on a set of independently sampled units. In the absence of an independent random sample, a relevant question is whether or not a statistical model that assumes such row exchangeability is plausible. One method for assessing this plausibility is a statistical test of row covariation. Maintenance of a constant type I erro…
Multivariate hypergeometric distribution arises frequently in elementary statistics and probability courses, for simultaneously studying the occurence law of specified events, when sampling without replacement from a finite population with fixed number of classification. Covariance matrix of this distribution is well known to be identical to its multinomial counterpart multiplied by 1-(n-1)/(N-1), with N and n being population and sample sizes, respectively. It appears to however, have been less discussed in the literature about the meaning of this relationship, especially regarding the specif…