Covariance matrix estimation is a fundamental statistical task in many applications, but the sample covariance matrix is sub-optimal when the sample size is comparable to or less than the number of features. Such high-dimensional settings are common in modern genomics, where covariance matrix estimation is frequently employed as a method for inferring gene networks. To achieve estimation accuracy in these settings, existing methods typically either assume that the population covariance matrix has some particular structure, for example sparsity, or apply shrinkage to better estimate the population eigenvalues. In this paper, we study a new approach to estimating high-dimensional covariance matrices. We first frame covariance matrix estimation as a compound decision problem. This motivates defining a class of decision rules and using a nonparametric empirical Bayes g-modeling approach to estimate the optimal rule in the class. Simulation results and gene network inference in an RNA-seq experiment in mouse show that our approach is comparable to or can outperform a number of state-of-the-art proposals.
We study the problem of incorporating covariates in a compound decision setup. It is desired to estimate the means of $n$ response variables, which are independent and normally distributed, and each is accompanied by a vector of covariates. We suggest a method that involves non-parametric empirical Bayes techniques and may be viewed as a generalization of the celebrated Fay-Herriot (1979) method. Some optimality properties of our method are proved. We also compare it numerically with Fay-Herriot and other methods, using a `semi-real' data set that involves spatio-temporal covariates, where the…
We discuss Bayesian inference for a known-mean Gaussian model with a compound symmetric variance-covariance matrix. Since the space of such matrices is a linear subspace of that of positive definite matrices, we utilize the methods of Pisano (2022) to decompose the usual Wishart conjugate prior and derive a closed-form, three-parameter, bivariate conjugate prior distribution for the compound-symmetric half-precision matrix. The off-diagonal entry is found to have a non-central Kummer-Beta distribution conditioned on the diagonal, which is shown to have a gamma distribution generalized with Gau…
We undertake Bayesian learning of the high-dimensional functional relationship between a system parameter vector and an observable, that is in general tensor-valued. The ultimate aim is Bayesian inverse prediction of the system parameters, at which test data is recorded. We attempt such learning given hypercuboidally-shaped data that displays strong discontinuities, rendering learning challenging. We model the sought high-dimensional function, with a tensor-variate Gaussian Process (GP), and use three independent ways for learning covariance matrices of the resulting likelihood, which is Tenso…
This paper considers the detection of change points in parallel data streams, a problem widely encountered when analyzing large-scale real-time streaming data. Each stream may have its own change point, at which its data has a distributional change. With sequentially observed data, a decision maker needs to declare whether changes have already occurred to the streams at each time point.Once a stream is declared to have changed, it is deactivated permanently so that its future data will no longer be collected. This is a compound decision problem in the sense that the decision maker may want to …
There is an innate human tendency, one might call it the "league table mentality," to construct rankings. Schools, hospitals, sports teams, movies, and myriad other objects are ranked even though their inherent multi-dimensionality would suggest that -- at best -- only partial orderings were possible. We consider a large class of elementary ranking problems in which we observe noisy, scalar measurements of merit for $n$ objects of potentially heterogeneous precision and are asked to select a group of the objects that are "most meritorious." The problem is naturally formulated in the compound d…
Based on a generalized cosine measure between two symmetric matrices, we propose a general framework for one-sample and two-sample tests of covariance and correlation matrices. We also develop a set of associated permutation algorithms for some common one-sample tests, such as the tests of sphericity, identity and compound symmetry, and the $K$-sample tests of multivariate equality of covariance or correlation matrices. The proposed method is very flexible in the sense that it does not assume any underlying distributions and data generation models. Moreover, it allows data to have different ma…
In standardized educational testing, test items are reused in multiple test administrations. To ensure the validity of test scores, the psychometric properties of items should remain unchanged over time. In this paper, we consider the sequential monitoring of test items, in particular, the detection of abrupt changes to their psychometric properties, where a change can be caused by, for example, leakage of the item or change of the corresponding curriculum. We propose a statistical framework for the detection of abrupt changes in individual items. This framework consists of (1) a multi-stream …
We address high dimensional covariance estimation for elliptical distributed samples, which are also known as spherically invariant random vectors (SIRV) or compound-Gaussian processes. Specifically we consider shrinkage methods that are suitable for high dimensional problems with a small number of samples (large $p$ small $n$). We start from a classical robust covariance estimator [Tyler(1987)], which is distribution-free within the family of elliptical distribution but inapplicable when $n<p$. Using a shrinkage coefficient, we regularize Tyler's fixed point iterations. We prove that, for all…
Covariance-Based Structural Equation Modeling in Small-Sample Settings with $p>n$
Graduate School of Science and Technology, University of Tsukuba（筑波大学科学技术研究院）；Institute of Systems and Information Engineering, University of Tsukuba（筑波大学系统与信息工程研究所）；Tsukuba Institute for Advanced Research, University of Tsukuba（筑波大学先进研究所）；Center for Artificial Intelligence Research, University of Tsukuba（筑波大学人工智能研究中心）
Informative sampling designs can impact spatial prediction, or kriging, in two important ways. First, the sampling design can bias spatial covariance parameter estimation, which in turn can bias spatial kriging estimates. Second, even with unbiased estimates of the spatial covariance parameters, since the kriging variance is a function of the observation locations, these estimates will vary based on the sample and overestimate the population-based estimates. In this work, we develop a weighted composite likelihood approach to improve spatial covariance parameter estimation under informative sa…
In unconstrained maximum a posteriori (MAP) and maximum likelihood estimation, the inverse of minus the merit-function Hessian matrix is an approximation of the estimate covariance matrix. In the Bayesian context of MAP estimation, it is the covariance of a normal approximation of the posterior around the mode; while in maximum likelihood estimation, it an approximation of the inverse Fisher information matrix, to which the covariance of efficient estimators converge. These measures are routinely used in system identification to evaluate the estimate uncertainties and diagnose problems such as…
