In this paper we investigate the performance of periodogram based estimators of the spectral density matrix of possibly high-dimensional time series. We suggest and study shrinkage as a remedy against numerical instabilities due to deteriorating condition numbers of (kernel) smoothed periodogram matrices. Moreover, shrinking the empirical eigenvalues in the frequency domain towards one another also improves at the same time the Mean Squared Error (MSE) of these widely used nonparametric spectral estimators. Compared to some existing time domain approaches, restricted to i.i.d. data, in the frequency domain it is necessary to take the size of the smoothing span as "effective or local sample size" into account. While Böhm and von Sachs (2007) proposes a multiple of the identity matrix as optimal shrinkage target in the absence of knowledge about the multidimensional structure of the data, here we consider "structural" shrinkage. We assume that the spectral structure of the data is induced by underlying factors. However, in contrast to actual factor modelling suffering from the need to choose the number of factors, we suggest a model-free approach. Our final estimator is the asymptotically MSE-optimal linear combination of the smoothed periodogram and the parametric estimator based on an underfitting (and hence deliberately misspecified) factor model. We complete our theoretical considerations by some extensive simulation studies. In the situation of data generated from a higher-order factor model, we compare all four types of involved estimators (including the one of Böhm and von Sachs (2007)).
Hawkes (1971) introduced a powerful multivariate point process model of mutually exciting processes to explain causal structure in data. In this paper it is shown that the Granger causality structure of such processes is fully encoded in the corresponding link functions of the model. A new nonparametric estimator of the link functions based on a time-discretized version of the point process is introduced by using an infinite order autoregression. Consistency of the new estimator is derived. The estimator is applied to simulated data and to neural spike train data from the spinal dorsal horn of…
We formulate nonparametric and semiparametric hypothesis testing of multivariate stationary linear time series in a unified fashion and propose new test statistics based on estimators of the spectral density matrix. The limiting distributions of these test statistics under null hypotheses are always normal distributions, and they can be implemented easily for practical use. If null hypotheses are false, as the sample size goes to infinity, they diverge to infinity and consequently are consistent tests for any alternative. The approach can be applied to various null hypotheses such as the indep…
This paper is concerned with modeling the dependence structure of two (or more) time-series in the presence of a (possible multivariate) covariate which may include past values of the time series. We assume that the covariate influences only the conditional mean and the conditional variance of each of the time series but the distribution of the standardized innovations is not influenced by the covariate and is stable in time. The joint distribution of the time series is then determined by the conditional means, the conditional variances and the marginal distributions of the innovations, which …
In this paper we propose a nonparametric procedure for validating the assumption of stationarity in multivariate locally stationary time series models. We develop a bootstrap assisted test based on a Kolmogorov-Smirnov type statistic, which tracks the deviation of the time varying spectral density from its best stationary approximation. In contrast to all other nonparametric approaches, which have been proposed in the literature so far, the test statistic does not depend on any regularization parameters like smoothing bandwidths or a window length, which is usually required in a segmentation o…
Analyzing large samples of high-dimensional data under dependence is a challenging statistical problem as long time series may have change points, most importantly in the mean and the marginal covariances, for which one needs valid tests. Inference for large covariance matrices is especially difficult due to noise accumulation, resulting in singular estimates and poor power of related tests. The singularity of the sample covariance matrix in high dimensions can be overcome by considering a linear combination with a regular, more structured target matrix. This approach is known as shrinkage, an…
We develop a new statistical method for estimating functional connectivity between neurophysiological signals represented by a multivariate time series. We use partial coherence as the measure of functional connectivity. Partial coherence identifies the frequency bands that drive the direct linear association between any pair of channels. To estimate partial coherence, one would first need an estimate of the spectral density matrix of the multivariate time series. Parametric estimators of the spectral density matrix provide good frequency resolution but could be sensitive when the parametric m…
We consider nonparametric Bayesian estimation and prediction for nonhomogeneous Poisson process models with unknown intensity functions. We propose a class of improper priors for intensity functions. Nonparametric Bayesian inference with kernel mixture based on the class improper priors is shown to be useful, although improper priors have not been widely used for nonparametric Bayes problems. Several theorems corresponding to those for finite-dimensional independent Poisson models hold for nonhomogeneous Poisson process models with infinite-dimensional parameter spaces. Bayesian estimation and…
We propose a flexible class of models based on scale mixture of uniform distributions to construct shrinkage priors for covariance matrix estimation. This new class of priors enjoys a number of advantages over the traditional scale mixture of normal priors, including its simplicity and flexibility in characterizing the prior density. We also exhibit a simple, easy to implement Gibbs sampler for posterior simulation which leads to efficient estimation in high dimensional problems. We first discuss the theory and computational details of this new approach and then extend the basic model to a new…
In this article we provide some nonnegative and positive estimators of the mean squared errors(MSEs) for shrinkage estimators of multivariate normal means. Proposed estimators are shown to improve on the uniformly minimum variance unbiased estimator(UMVUE) under a quadratic loss criterion. A similar improvement is also obtained for the estimators of the MSE matrices for shrinkage estimators. We also apply the proposed estimators of the MSE matrix to form confidence sets centered at shrinkage estimators and show their usefulness through numerical experiments.
The problem of estimating a mean matrix of a multivariate complex normal distribution with an unknown covariance matrix is considered under an invariant loss function. By using complex versions of the Stein identity, the Stein-Haff identity, and calculus on eigenvalues, a formula is obtained for an unbiased estimate of the risk of an invariant class of estimators, from which several minimax shrinkage estimators are constructed.
In this work, the estimation of the multivariate normal mean by different classes of shrinkage estimators is investigated. The risk associated with the balanced loss function is used to compare two estimators. We start by considering estimators that generalize the James-Stein estimator and show that these estimators dominate the maximum likelihood estimator (MLE), therefore are minimax, when the shrinkage function satisfies some conditions. Then, we treat estimators of polynomial form and prove the increase of the degree of the polynomial allows us to build a better estimator from the one prev…
The study of dependence between random variables is the core of theoretical and applied statistics. Static and dynamic copula models are useful for describing the dependence structure, which is fully encrypted in the copula probability density function. However, these models are not always able to describe the temporal change of the dependence patterns, which is a key characteristic of financial data. We propose a novel nonparametric framework for modelling a time series of copula probability density functions, which allows to forecast the entire function without the need of post-processing pr…
We propose computationally efficient methods for estimating stationary multivariate spatial and spatial-temporal spectra from incomplete gridded data. The methods are iterative and rely on successive imputation of data and updating of model estimates. Imputations are done according to a periodic model on an expanded domain. The periodicity of the imputations is a key feature that reduces edge effects in the periodogram and is facilitated by efficient circulant embedding techniques. In addition, we describe efficient methods for decomposing the estimated cross spectral density function into a l…
We introduce a hierarchical nonparametric model for probability measures based on a multi-resolution transformation of probability distributions. The model allows a varying amount of shrinkage to be applied to data features of different scales and/or at different locations in the sample space, and the varying shrinkage level is locally adaptive to the empirical behavior of the data. Moreover, the model's hierarchical design---through a latent Markov tree structure---allows borrowing of information across locations and scales in setting the adaptive shrinkage level. Inference under the model pr…
In recent years, shrinkage priors have received much attention in high-dimensional data analysis from a Bayesian perspective. Compared with widely used spike-and-slab priors, shrinkage priors have better computational efficiency. But the theoretical properties, especially posterior contraction rate, which is important in uncertainty quantification, are not established in many cases. In this paper, we apply global-local shrinkage priors to high-dimensional multivariate linear regression with unknown covariance matrix. We show that when the prior is highly concentrated near zero and has heavy ta…
In this article, we consider two forms of shrinkage estimators of the mean $θ$ of a multivariate normal distribution $X\sim N_{p}\left(θ, σ^{2}I_{p}\right)$ where $σ^{2}$ is unknown. We take the prior law $θ\sim N_{p}\left(\upsilon, τ^{2}I_{p}\right)$ and we constuct a Modified Bayes estimator $δ_{B}^{\ast}$ and an Empirical Modified Bayes estimator $δ_{EB}^{\ast}$. We are interested in studying the minimaxity and the limits of risks ratios of these estimators, to the maximum likelihood estimator $X$, when $n$ and $p$ tend to infinity.
A macaque monkey is trained to perform two different kinds of tasks, memory aided and visually aided. In each task, the monkey saccades to eight possible target locations. A classifier is proposed for direction decoding and task decoding based on local field potentials (LFP) collected from the prefrontal cortex. The LFP time-series data is modeled in a nonparametric regression framework, as a function corrupted by Gaussian noise. It is shown that if the function belongs to Besov bodies, then using the proposed wavelet shrinkage and thresholding based classifier is robust and consistent. The cl…
We study shrinkage estimation of the mean parameters of a class of multivariate distributions for which the diagonal entries of the corresponding covariance matrix are certain quadratic functions of the mean parameter. This class of distributions includes the diagonal multivariate natural exponential families. We propose two classes of semi-parametric shrinkage estimators for the mean and construct unbiased estimators of the corresponding risk. We establish the asymptotic consistency and convergence rates for these shrinkage estimators under squared error loss as both $n$, the sample size, and…
We show that nonparametric regression is asymptotically equivalent in Le Cam's sense with a sequence of Gaussian white noise experiments as the number of observations tends to infinity. We propose a general constructive framework based on approximation spaces, which permits to achieve asymptotic equivalence even in the cases of multivariate and random design.
We investigate predictive densities for multivariate normal models with unknown mean vectors and known covariance matrices. Bayesian predictive densities based on shrinkage priors often have complex representations, although they are effective in various problems. We consider extended normal models with mean vectors and covariance matrices as parameters, and adopt predictive densities that belong to the extended models including the original normal model. We adopt predictive densities that are optimal with respect to the posterior Bayes risk in the extended models. The proposed predictive dens…
This paper studies nonparametric estimation of parameters of multivariate Hawkes processes. We consider the Bayesian setting and derive posterior concentration rates. First rates are derived for L1-metrics for stochastic intensities of the Hawkes process. We then deduce rates for the L1-norm of interactions functions of the process. Our results are exemplified by using priors based on piecewise constant functions, with regular or random partitions and priors based on mixtures of Betas distributions. Numerical illustrations are then proposed with in mind applications for inferring functional co…
We consider a continuous-time stochastic volatility model. The model contains a stationary volatility process, the multivariate density of the finite dimensional distributions of which we aim to estimate. We assume that we observe the process at discrete instants in time. The sampling times will be equidistant with vanishing distance. A multivariate Fourier-type deconvolution kernel density estimator based on the logarithm of the squared processes is proposed to estimate the multivariate volatility density. An expansion of the bias and a bound on the variance are derived. Key words: stochastic…
We consider the nonparametric estimation problem of time-dependent multivariate functions observed in a presence of additive cylindrical Gaussian white noise of a small intensity. We derive minimax lower bounds for the $L^2$-risk in the proposed spatio-temporal model as the intensity goes to zero, when the underlying unknown response function is assumed to belong to a ball of appropriately constructed inhomogeneous time-dependent multivariate functions, motivated by practical applications. Furthermore, we propose both non-adaptive linear and adaptive non-linear wavelet estimators that are asym…
We provide a nonparametric method for the computation of instantaneous multivariate volatility for continuous semi-martingales, which is based on Fourier analysis. The co-volatility is reconstructed as a stochastic function of time by establishing a connection between the Fourier transform of the prices process and the Fourier transform of the co-volatility process. A nonparametric estimator is derived given a discrete unevenly spaced and asynchronously sampled observations of the asset price processes. The asymptotic properties of the random estimator are studied: namely, consistency in proba…
This paper deals with the consistency of the least squares estimator of a convex regression function when the predictor is multidimensional. We characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs. Mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided. We also consider a regression function which is known to be convex and componentwise nonincreasing and discuss the characterization, computation and consistency of its least square…
In this paper, we consider the nonparametric regression problem with multivariate predictors. We provide a characterization of the degrees of freedom and divergence for estimators of the unknown regression function, which are obtained as outputs of linearly constrained quadratic optimization procedures, namely, minimizers of the least squares criterion with linear constraints and/or quadratic penalties. As special cases of our results, we derive explicit expressions for the degrees of freedom in many nonparametric regression problems, e.g., bounded isotonic regression, multivariate (penalized)…
We study concentration in spectral norm of nonparametric estimates of correlation matrices. We work within the confine of a Gaussian copula model. Two nonparametric estimators of the correlation matrix, the sine transformations of the Kendall's tau and Spearman's rho correlation coefficient, are studied. Expected spectrum error bound is obtained for both the estimators. A general large deviation bound for the maximum spectral error of a collection of submatrices of a given dimension is also established. These results prove that when both the number of variables and sample size are large, the s…
Density estimation represents one of the most successful applications of Bayesian nonparametrics. In particular, Dirichlet process mixtures of normals are the gold standard for density estimation and their asymptotic properties have been studied extensively, especially in the univariate case. However a gap between practitioners and the current theoretical literature is present. So far, posterior asymptotic results in the multivariate case are available only for location mixtures of Gaussian kernels with independent prior on the common covariance matrix, while in practice as well as from a conc…
Although continuous density estimation has received abundant attention in the Bayesian nonparametrics literature, there is limited theory on multivariate mixed scale density estimation. In this note, we consider a general framework to jointly model continuous, count and categorical variables under a nonparametric prior, which is induced through rounding latent variables having an unknown density with respect to Lebesgue measure. For the proposed class of priors, we provide sufficient conditions for large support, strong consistency and rates of posterior contraction. These conditions allow one…
The estimation of a multivariate mean $θ$ is considered under natural modifications of balanced loss function of the form: (i) $ω\, ρ(\|δ-δ_0\|^2) + (1-ω) \, ρ(\|δ-θ\|^2) $, and (ii) $\ell \left( ω\, \|δ-δ_0\|^2 + (1-ω) \, \|δ-θ\|^2 \right)\,$, where $δ_0$ is a target estimator of $γ(θ)$. After briefly reviewing known results for original balanced loss with identity $ρ$ or $\ell$, we provide, for increasing and concave $ρ$ and $\ell$ which also satisfy a completely monotone property, Baranchik-type estimators of $θ$ which dominate the benchmark $δ_0(X)=X$ for $X$ either distributed as multivar…
In this paper, a novel Bayesian nonparametric test for assessing multivariate normal models is presented. While there are extensive frequentist and graphical methods for testing multivariate normality, it is challenging to find Bayesian counterparts. The proposed approach is based on the use of the Dirichlet process and Mahalanobis distance. More precisely, the Mahalanobis distance is employed as a good technique to transform the $m$-variate problem into a univariate problem. Then the Dirichlet process is used as a prior on the distribution of the Mahalanobis distance. The concentration of the…
In this paper, we propose novel, fully Bayesian non-parametric tests for one-sample and two-sample multivariate location problems. We model the underlying distribution using a Dirichlet process prior, and develop a testing procedure based on the posterior credible region for the spatial median functional of the distribution. For the one-sample problem, we fail to reject the null hypothesis if the credible set contains the null value. For the two-sample problem, we form a credible set for the difference of the spatial medians of the two samples and we fail to reject the null hypothesis of equal…
There has been a wide interest to extend univariate and multivariate nonparametric procedures to clustered and hierarchical data. Traditionally, parametric mixed models have been used to account for the correlation structures among the dependent observational units. In this work we extend multivariate nonparametric procedures for one-sample and several samples location problems to clustered data settings. The results are given for a general score function, but with an emphasis on spatial sign and rank methods. Mixed models notation involving design matrices for fixed and random effects is used…
This paper introduces and studies multivariate spacings. The spacings are developed using the order statistics derived from data depth. Specifically, the spacing between two consecutive order statistics is the region which bridges the two order statistics, in the sense that the region contains all the points whose depth values fall between the depth values of the two consecutive order statistics. These multivariate spacings can be viewed as a data-driven realization of the so-called ``statistically equivalent blocks.'' These spacings assume a form of center-outward layers of ``shells'' (``ring…
In this paper, we propose a theoretical analysis of the algorithm ISDE, introduced in previous work. From a dataset, ISDE learns a density written as a product of marginal density estimators over a partition of the features. We show that under some hypotheses, the Kullback-Leibler loss between the proper density and the output of ISDE is a bias term plus the sum of two terms which goes to zero as the number of samples goes to infinity. The rate of convergence indicates that ISDE tackles the curse of dimensionality by reducing the dimension from the one of the ambient space to the one of the bi…
Multivariate analysis of variance (MANOVA) is a powerful and versatile method to infer and quantify main and interaction effects in metric multivariate multi-factor data. It is, however, neither robust against change in units nor a meaningful tool for ordinal data. Thus, we propose a novel nonparametric MANOVA. Contrary to existing rank-based procedures we infer hypotheses formulated in terms of meaningful Mann-Whitney-type effects in lieu of distribution functions. The tests are based on a quadratic form in multivariate rank effect estimators and critical values are obtained by the bootstrap.…
In this paper, a nonparametric estimator is proposed for estimating the L1-median for multivariate conditional distribution when the covariates take values in an infinite dimensional space. The multivariate case is more appropriate to predict the components of a vector of random variables simultaneously rather than predicting each of them separately. While estimating the conditional L1-median function using the well-known Nadarya-Waston estimator, we establish the strong consistency of this estimator as well as the asymptotic normality. We also present some simulations and provide how to built…
In the setting of nonparametric multivariate regression with unknown error variance, we study asymptotic properties of a Bayesian method for estimating a regression function f and its mixed partial derivatives. We use a random series of tensor product of B-splines with normal basis coefficients as a prior for f, and the error variance is either estimated using the empirical Bayes approach or is endowed with a suitable prior in a hierarchical Bayes approach. We establish pointwise, L2 and supremum norm posterior contraction rates for f and its mixed partial derivatives, and show that they coinc…
Bayesian methods are developed for the multivariate nonparametric regression problem where the domain is taken to be a compact Riemannian manifold. In terms of the latter, the underlying geometry of the manifold induces certain symmetries on the multivariate nonparametric regression function. The Bayesian approach then allows one to incorporate hierarchical Bayesian methods directly into the spectral structure, thus providing a symmetry-adaptive multivariate Bayesian function estimator. One can also diffuse away some prior information in which the limiting case is a smoothing spline on the man…
The estimation of the mean matrix of the multivariate normal distribution is addressed in the high dimensional setting. Efron-Morris-type linear shrinkage estimators based on ridge estimators for the precision matrix instead of the Moore-Penrose generalized inverse are considered, and the weights in the ridge-type linear shrinkage estimators are estimated in terms of minimizing the Stein unbiased risk estimators under the quadratic loss. It is shown that the ridge-type linear shrinkage estimators with the estimated weights are minimax, and that the estimated weights converge to the optimal wei…
In this work, we study a random orthogonal projection based least squares estimator for the stable solution of a multivariate nonparametric regression (MNPR) problem. More precisely, given an integer $d\geq 1$ corresponding to the dimension of the MNPR problem, a positive integer $N\geq 1$ and a real parameter $α\geq -\frac{1}{2},$ we show that a fairly large class of $d-$variate regression functions are well and stably approximated by its random projection over the orthonormal set of tensor product $d-$variate Jacobi polynomials with parameters $(α,α).$ The associated uni-variate Jacobi polyn…
In this paper we study multivariate ranks and quantiles, defined using the theory of optimal transport, and build on the work of Chernozhukov et al.(2017) and Hallin et al.(2021). We study the characterization, computation and properties of the multivariate rank and quantile functions and their empirical counterparts. We derive the uniform consistency of these empirical estimates to their population versions, under certain assumptions. In fact, we prove a Glivenko-Cantelli type theorem that shows the asymptotic stability of the empirical rank map in any direction. Under mild structural assumpt…
Although unbiasedness is a basic property of a good test, many tests on vector parameters or scalar parameters against two-sided alternatives are not finite-sample unbiased. This was already noticed by Sugiura [Ann. Inst. Statist. Math. 17 (1965) 261--263]; he found an alternative against which the Wilcoxon test is not unbiased. The problem is even more serious in multivariate models. When testing the hypothesis against an alternative which fits well with the experiment, it should be verified whether the power of the test under this alternative cannot be smaller than the significance level. Su…
In this paper, we propose a general framework for distribution-free nonparametric testing in multi-dimensions, based on a notion of multivariate ranks defined using the theory of measure transportation. Unlike other existing proposals in the literature, these multivariate ranks share a number of useful properties with the usual one-dimensional ranks; most importantly, these ranks are distribution-free. This crucial observation allows us to design nonparametric tests that are exactly distribution-free under the null hypothesis. We demonstrate the applicability of this approach by constructing e…
We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The ``COSSO'' is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the …
We consider the problem of adaptive inference on a regression function at a point under a multivariate nonparametric regression setting. The regression function belongs to a Hölder class and is assumed to be monotone with respect to some or all of the arguments. We derive the minimax rate of convergence for confidence intervals (CIs) that adapt to the underlying smoothness, and provide an adaptive inference procedure that obtains this minimax rate. The procedure differs from that of Cai and Low (2004), intended to yield shorter CIs under practically relevant specifications. The proposed method…
