This paper considers the estimation and inference of the low-rank components in high-dimensional matrix-variate factor models, where each dimension of the matrix-variates ($p \times q$) is comparable to or greater than the number of observations ($T$). We propose an estimation method called $α$-PCA that preserves the matrix structure and aggregates mean and contemporary covariance through a hyper-parameter $α$. We develop an inferential theory, establishing consistency, the rate of convergence, and the limiting distributions, under general conditions that allow for correlations across time, rows, or columns of the noise. We show both theoretical and empirical methods of choosing the best $α$, depending on the use-case criteria. Simulation results demonstrate the adequacy of the asymptotic results in approximating the finite sample properties. The $α$-PCA compares favorably with the existing ones. Finally, we illustrate its applications with a real numeric data set and two real image data sets. In all applications, the proposed estimation procedure outperforms previous methods in the power of variance explanation using out-of-sample 10-fold cross-validation.
Matrix variate beta (MVB) distributions are used in different fields of hypothesis testing, multivariate correlation analysis, zero regression, canonical correlation analysis and etc. In this approach a unified methodology is proposed to generate matrix variate distributions by combining the kernel of MVB distributions of different types with an unknown Borel measurable function of trace operator over matrix space, called generator component. The latter component is a principal element of these newly defined generator type matrix variate distributions. The matrix variate Kummer beta distributi…
Statistical analysis of tensor-valued data has largely used the tensor-variate normal (TVN) distribution that may be inadequate when data comes from distributions with heavier or lighter tails. We study a general family of elliptically contoured (EC) tensor-variate distributions and derive its characterizations, moments, marginal and conditional distributions, and the EC Wishart distribution. We describe procedures for maximum likelihood estimation from data that are (1) uncorrelated draws from an EC distribution, (2) from a scale mixture of the TVN distribution, and (3) from an underlying but…
This paper studies the Gaussian approximation of high-dimensional and non-degenerate U-statistics of order two under the supremum norm. We propose a two-step Gaussian approximation procedure that does not impose structural assumptions on the data distribution. Specifically, subject to mild moment conditions on the kernel, we establish the explicit rate of convergence that decays polynomially in sample size for a high-dimensional scaling limit, where the dimension can be much larger than the sample size. We also supplement a practical Gaussian wild bootstrap method to approximate the quantiles …
The noncentral Wishart distribution has become more mainstream in statistics as the prevalence of applications involving sample covariances with underlying multivariate Gaussian populations as dramatically increased since the advent of computers. Multiple sources in the literature deal with local approximations of the noncentral Wishart distribution with respect to its central counterpart. However, no source has yet developed explicit local approximations for the (central) Wishart distribution in terms of a normal analogue, which is important since Gaussian distributions are at the heart of th…
Many predictions are probabilistic in nature; for example, a prediction could be for precipitation tomorrow, but with only a 30 percent chance. Given both the predictions and the actual outcomes, "reliability diagrams" (also known as "calibration plots") help detect and diagnose statistically significant discrepancies between the predictions and the outcomes. The canonical reliability diagrams are based on histogramming the observed and expected values of the predictions; several variants of the standard reliability diagrams propose to replace the hard histogram binning with soft kernel densit…
We estimate the parameter of a stationary time series process by minimizing the integrated weighted mean squared error between the empirical and simulated characteristic function, when the true characteristic functions cannot be explicitly computed. Motivated by Indirect Inference, we use a Monte Carlo approximation of the characteristic function based on iid simulated blocks. As a classical variance reduction technique, we propose the use of control variates for reducing the variance of this Monte Carlo approximation. These two approximations yield two new estimators that are applicable to a …
The upsilon distribution, the sum of independent chi random variates and a normal, is introduced. As a special case, the upsilon distribution includes Lecoutre's lambda-prime distribution. The upsilon distribution finds application in Frequentist inference on the Sharpe ratio, including hypothesis tests on independent samples, confidence intervals, and prediction intervals, as well as their Bayesian counterparts. These tests are extended to the case of factor models of returns.
This paper presents a selective survey of recent developments in statistical inference and multiple testing for high-dimensional regression models, including linear and logistic regression. We examine the construction of confidence intervals and hypothesis tests for various low-dimensional objectives such as regression coefficients and linear and quadratic functionals. The key technique is to generate debiased and desparsified estimators for the targeted low-dimensional objectives and estimate their uncertainty. In addition to covering the motivations for and intuitions behind these statistica…
This paper is dedicated to the "50 Years of the Relevance Problem" - a long-neglected topic that begs attention from practical statisticians who are concerned with the problem of drawing inference from large-scale heterogeneous data.
Laplace's method is used to approximate intractable integrals in a statistical problems. The relative error rate of the approximation is not worse than $O_p(n^{-1})$. We provide the first statistical lower bounds showing that the $n^{-1}$ rate is tight.
Statistical pragmatism embraces all efficient methods in statistical inference. Augmentation of the collected data is used herein to obtain representative population information from a large class of non-representative population's units. Parameter expansion of a probability model is shown to reduce the upper bound on the sum of error probabilities for a test of simple hypotheses, and a measure, R, is proposed for the effect of activating additional component(s) in the sufficient statistic.
We wholeheartedly congratulate Drs. Rohe and Zeng for their insightful paper \cite{rohe2020vintage} on vintage factor analysis with Varimax rotation. This note discusses the conditions to guarantee Varimax consistently recovers the subspace rotation.
Statistical models and methods for determinantal point processes (DPPs) seem largely unexplored. We demonstrate that DPPs provide useful models for the description of spatial point pattern datasets where nearby points repel each other. Such data are usually modelled by Gibbs point processes, where the likelihood and moment expressions are intractable and simulations are time consuming. We exploit the appealing probabilistic properties of DPPs to develop parametric models, where the likelihood and moment expressions can be easily evaluated and realizations can be quickly simulated. We discuss h…
This paper considers the problem of cardinality estimation in data stream applications. We present a statistical analysis of probabilistic counting algorithms, focusing on two techniques that use pseudo-random variates to form low-dimensional data sketches. We apply conventional statistical methods to compare probabilistic algorithms based on storing either selected order statistics, or random projections. We derive estimators of the cardinality in both cases, and show that the maximal-term estimator is recursively computable and has exponentially decreasing error bounds. Furthermore, we show …
Geostatistical modeling for continuous point-referenced data has been extensively applied to neuroimaging because it produces efficient and valid statistical inference. However, diffusion tensor imaging (DTI), a neuroimaging characterizing the brain structure produces a positive definite (p.d.) matrix for each voxel. Current geostatistical modeling has not been extended to p.d. matrices because introducing spatial dependence among positive definite matrices properly is challenging. In this paper, we use the spatial Wishart process, a spatial stochastic process (random field) where each p.d. ma…
The topic of statistical inference for dynamical systems has been studied extensively across several fields. In this survey we focus on the problem of parameter estimation for non-linear dynamical systems. Our objective is to place results across distinct disciplines in a common setting and highlight opportunities for further research.
Integrated organic inference (IOI) is discussed in a concise and informal way with the aim that the reader is given the gist of what this approach to statistical inference is about as well as given pointers to further reading.
We consider a new method of the semiparametric statistical estimation for the continuous-time moving average Lévy processes. We derive the convergence rates of the proposed estimators, and show that these rates are optimal in the minimax sense.
The Wald development of statistical decision theory addresses decision making with sample data. Wald's concept of a statistical decision function (SDF) embraces all mappings of the form [data -> decision]. An SDF need not perform statistical inference; that is, it need not use data to draw conclusions about the true state of nature. Inference-based SDFs have the sequential form [data -> inference -> decision]. This paper motivates inference-based SDFs as practical procedures for decision making that may accomplish some of what Wald envisioned. The paper first addresses binary choice problems, …
Statistics has moved beyond the frequentist-Bayesian controversies of the past. Where does this leave our ability to interpret results? I suggest that a philosophy compatible with statistical practice, labeled here statistical pragmatism, serves as a foundation for inference. Statistical pragmatism is inclusive and emphasizes the assumptions that connect statistical models with observed data. I argue that introductory courses often mischaracterize the process of statistical inference and I propose an alternative "big picture" depiction.
Standard statistical methods applied to matrix random variables often fail to describe the underlying structure in multiway data sets. In this paper we will discuss the concept of an array variate random variable and introduce a class of elliptical array densities which have elliptical contours.
Introductory statistical inference texts and courses treat the point estimation, hypothesis testing, and interval estimation problems separately, with primary emphasis on large-sample approximations. Here I present an alternative approach to teaching this course, built around p-values, emphasizing provably valid inference for all sample sizes. Details about computation and marginalization are also provided, with several illustrative examples, along with a course outline.
We consider the problem of large-scale inference on the row or column variables of data in the form of a matrix. Often this data is transposable, meaning that both the row variables and column variables are of potential interest. An example of this scenario is detecting significant genes in microarrays when the samples or arrays may be dependent due to underlying relationships. We study the effect of both row and column correlations on commonly used test-statistics, null distributions, and multiple testing procedures, by explicitly modeling the covariances with the matrix-variate normal distri…
From the climate system to the effect of the internet on society, chaotic systems appear to have a significant role in our future. Here a method of statistical learning for a class of chaotic systems is described along with underlying theory that can be used not only for predicting those systems a short time ahead, but also as a basis for statistical inference about their dynamics. The method is applied to prediction of 3 different systems. The statistical inference aspect can be applied to explore and enhance computer models of such systems which in turn can provide feedback for even better p…
Nagoya Institute of Technology（名古屋工业大学）；RIKEN（日本理化学研究所）
In this paper, we study statistical inference of change-points (CPs) in multi-dimensional sequence. In CP detection from a multi-dimensional sequence, it is often desirable not only to detect the location, but also to identify the subset of the components in which the change occurs. Several algorithms have been proposed for such problems, but no valid exact inference method has been established to evaluate the statistical reliability of the detected locations and components. In this study, we propose a method that can guarantee the statistical reliability of both the location and the component…
Probability distributions in Stiefel manifold such as the von-Mises Fisher and Bingham distributions find diverse applications in signal processing and other applied sciences. Use of these statistical models in practice is complicated by the difficulties in numerical evaluation of their normalization constants. In this letter, we derive asymptotical approximations to the normalization constants via recent results in random matrix theory. The derived approximations take simple forms and are reasonably accurate in regimes of practical interest. As an application, we show that the proposed analyt…
This document is the Part IV of the book 'Stochastic Epidemic Models with Inference' edited by Tom Britton and Etienne Pardoux. It is written by Catherine Larédo, with the contribution of Viet Chi Tran for the Chapter 4. Epidemic data present challenging statistical problems, starting from the recurrent issue of handling missing information. We review methods such as MCMC, ABC or methods based on diffusion approximations. Plan of this document: 1) Observations and Asymptotic Frameworks; 2) Inference for Markov Chain Epidemic Models; 3) Inference Based on the Diffusion Approximation of Epidemic…
This paper is concerned with the application of recent statistical advances to inference of infectious disease dynamics. We describe the fitting of a class of epidemic models using Hamiltonian Monte Carlo and Variational Inference as implemented in the freely available Stan software. We apply the two methods to real data from outbreaks as well as routinely collected observations. Our results suggest that both inference methods are computationally feasible in this context, and show a trade-off between statistical efficiency versus computational speed. The latter appears particularly relevant fo…
Discovering statistically significant patterns from databases is an important challenging problem. The main obstacle of this problem is in the difficulty of taking into account the selection bias, i.e., the bias arising from the fact that patterns are selected from extremely large number of candidates in databases. In this paper, we introduce a new approach for predictive pattern mining problems that can address the selection bias issue. Our approach is built on a recently popularized statistical inference framework called selective inference. In selective inference, statistical inferences (su…
A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible MCMC samplers. We propose the use of a specific class of functions as control variates, and we introduce a new, consistent estimator for the values of the coefficients of the optimal linear combination of these functions. The form and proposed construction of the control variates is derived from our solution of the Poisson equation associated with a specific MCMC scenario. The new estimator, which can be applied to the same MCMC sample, is…
We consider the following data perturbation model, where the covariates incur multiplicative errors. For two $n \times m$ random matrices $U, X$, we denote by $U \circ X$ the Hadamard or Schur product, which is defined as $(U \circ X)_{ij} = (U_{ij}) \cdot (X_{ij})$. In this paper, we study the subgaussian matrix variate model, where we observe the matrix variate data $X$ through a random mask $U$: $$ {\mathcal X} = U \circ X \; \; \; \text{ where} \; \; \;X = B^{1/2} {\mathbb{Z}} A^{1/2}, $$ where ${\mathbb{Z}}$ is a random matrix with independent subgaussian entries, and $U$ is a mask matrix…
We introduce new estimation methods for a sub-class of the Gaussian scale mixture models for wavelet trees by Wainwright, Simoncelli & Willsky that rely on modern results for composite likelihoods and approximate Bayesian inference. Our methodology is illustrated for denoising and edge detection problems in two-dimensional images.
The features of a logically sound approach to a theory of statistical reasoning are discussed. A particular approach that satisfies these criteria is reviewed. This is seen to involve selection of a model, model checking, elicitation of a prior, checking the prior for bias, checking for prior-data conflict and estimation and hypothesis assessment inferences based on a measure of evidence. A long-standing anomalous example is resolved by this approach to inference and an application is made to a practical problem of considerable importance which, among other novel aspects of the analysis, invol…
Estimation of the $ϕ$-divergence between two unknown probability distributions using empirical data is a fundamental problem in information theory and statistical learning. We consider a multi-variate generalization of the data dependent partitioning method for estimating divergence between the two unknown distributions. Under the assumption that the distribution satisfies a power law of decay, we provide a convergence rate result for this method on the number of samples and hyper-rectangles required to ensure the estimation error is bounded by a given level with a given probability.
We introduce the concepts of Bayesian lens, characterizing the bidirectional structure of exact Bayesian inference, and statistical game, formalizing the optimization objectives of approximate inference problems. We prove that Bayesian inversions compose according to the compositional lens pattern, and exemplify statistical games with a number of classic statistical concepts, from maximum likelihood estimation to generalized variational Bayesian methods. This paper is the first in a series laying the foundations for a compositional account of the theory of active inference, and we therefore pa…
