This paper provides closed-form expansions for the log-likelihood function of multivariate diffusions sampled at discrete time intervals. The coefficients of the expansion are calculated explicitly by exploiting the special structure afforded by the diffusion model. Examples of interest in financial statistics and Monte Carlo evidence are included, along with the convergence of the expansion to the true likelihood function.
In this paper we consider a variety of procedures for numerical statistical inference in the family of univariate and multivariate stable distributions. In connection with univariate distributions (i) we provide approximations by finite location-scale mixtures and (ii) versions of approximate Bayesian computation (ABC) using the characteristic function and the asymptotic form of the likelihood function. In the context of multivariate stable distributions we propose several ways to perform statistical inference and obtain the spectral measure associated with the distributions, a quantity that h…
Spectral subsampling MCMC was recently proposed to speed up Markov chain Monte Carlo (MCMC) for long stationary univariate time series by subsampling periodogram observations in the frequency domain. This article extends the approach to multivariate time series using a multivariate generalisation of the Whittle likelihood. To assess the computational gains from spectral subsampling in challenging problems, a multivariate generalisation of the autoregressive tempered fractionally integrated moving average model (ARTFIMA) is introduced and some of its properties derived. Bayesian inference based…
The distributional transform (DT) is amongst the computational methods used for estimation of high-dimensional multivariate normal copula models with discrete responses. Its advantage is that the likelihood can be derived conveniently under the theory for copula models with continuous margins, but there has not been a clear analysis of the adequacy of this method. We investigate the small-sample and asymptotic efficiency of the method for estimating high-dimensional multivariate normal copula models with univariate Bernoulli, Poisson, and negative binomial margins, and show that the DT approxi…
The continuous extension of a discrete random variable is amongst the computational methods used for estimation of multivariate normal copula-based models with discrete margins. Its advantage is that the likelihood can be derived conveniently under the theory for copula models with continuous margins, but there has not been a clear analysis of the adequacy of this method. We investigate the asymptotic and small-sample efficiency of two variants of the method for estimating the multivariate normal copula with univariate binary, Poisson, and negative binomial regressions, and show that they lead…
An MCMC simulation method based on a two stage delayed rejection Metropolis-Hastings algorithm is proposed to estimate a factor multivariate stochastic volatility model. The first stage uses kstep iteration towards the mode, with k small, and the second stage uses an adaptive random walk proposal density. The marginal likelihood approach of Chib (1995) is used to choose the number of factors, with the posterior density ordinates approximated by Gaussian copula. Simulation and real data applications suggest that the proposed simulation method is computationally much more efficient than the appr…
Jing (1995) and Liu et al. (2008) studied the two-sample empirical likelihood and showed it is Bartlett correctable for the univariate and multivariate cases, respectively. We expand its domain to the full parameter space and obtain a two-sample extended empirical likelihood which is more accurate and can also achieve the second-order accuracy of the Bartlett correction.
In this paper we obtain an adjusted version of the likelihood ratio test for errors-in-variables multivariate linear regression models. The error terms are allowed to follow a multivariate distribution in the class of the elliptical distributions, which has the multivariate normal distribution as a special case. We derive a modified likelihood ratio statistic that follows a chi-squared distribution with a high degree of accuracy. Our results generalize those in Melo and Ferrari(Advances in Statistical Analysis, 2010, 94, 75-87) by allowing the parameter of interest to be vector-valued in the m…
Max-stable distributions and processes are important models for extreme events and the assessment of tail risks. The full, multivariate likelihood of a parametric max-stable distribution is complicated and only recent advances enable its use. The asymptotic properties of the maximum likelihood estimator in multivariate extremes are mostly unknown. In this paper we provide natural conditions on the exponent function and the angular measure of the max-stable distribution that ensure asymptotic normality of the estimator. We show the effectiveness of this result by applying it to popular parametr…
In this paper, we develop modified versions of the likelihood ratio test for multivariate heteroskedastic errors-in-variables regression models. The error terms are allowed to follow a multivariate distribution in the elliptical class of distributions, which has the normal distribution as a special case. We derive the Skovgaard adjusted likelihood ratio statistics, which follow a chi-squared distribution with a high degree of accuracy. We conduct a simulation study and show that the proposed tests display superior finite sample behavior as compared to the standard likelihood ratio test. We ill…
The multivariate probit model (MVP) is a popular classic model for studying binary responses of multiple entities. Nevertheless, the computational challenge of learning the MVP model, given that its likelihood involves integrating over a multidimensional constrained space of latent variables, significantly limits its application in practice. We propose a flexible deep generalization of the classic MVP, the Deep Multivariate Probit Model (DMVP), which is an end-to-end learning scheme that uses an efficient parallel sampling process of the multivariate probit model to exploit GPU-boosted deep ne…
The asymptotic normality of the maximum likelihood estimator (MLE) under regularity conditions is a cornerstone of statistical theory. In this paper, we give explicit upper bounds on the distributional distance between the distribution of the MLE of a vector parameter, and the multivariate normal distribution. We work with possibly high-dimensional, independent but not necessarily identically distributed random vectors. In addition, we obtain explicit upper bounds even in cases where the MLE cannot be expressed analytically.
We present new results for consistency of maximum likelihood estimators with a focus on multivariate mixed models. Our theory builds on the idea of using subsets of the full data to establish consistency of estimators based on the full data. It requires neither that the data consist of independent observations, nor that the observations can be modeled as a stationary stochastic process. Compared to existing asymptotic theory using the idea of subsets we substantially weaken the assumptions, bringing them closer to what suffices in classical settings. We apply our theory in two multivariate mix…
Reduced-rank regression is a dimensionality reduction method with many applications. The asymptotic theory for reduced rank estimators of parameter matrices in multivariate linear models has been studied extensively. In contrast, few theoretical results are available for reduced-rank multivariate generalised linear models. We develop M-estimation theory for concave criterion functions that are maximised over parameters spaces that are neither convex nor closed. These results are used to derive the consistency and asymptotic distribution of maximum likelihood estimators in reduced-rank multivar…
Robust estimation under multivariate normal (MVN) mixture model is always a computational challenge. A recently proposed maximum pseudo \b{eta}-likelihood estimator aims to estimate the unknown parameters of a MVN mixture model in the spirit of minimum density power divergence (DPD) methodology but with a relatively simpler and tractable computational algorithm even for larger dimensions. In this letter, we will rigorously derive the existence and weak consistency of the maximum pseudo \b{eta}-likelihood estimator in case of MVN mixture models under a reasonable set of assumptions.
We use the delta method and Stein's method to derive, under regularity conditions, explicit upper bounds for the distributional distance between the distribution of the maximum likelihood estimator (MLE) of a $d$-dimensional parameter and its asymptotic multivariate normal distribution. Our bounds apply in situations in which the MLE can be written as a function of a sum of i.i.d. $t$-dimensional random vectors. We apply our general bound to establish a bound for the multivariate normal approximation of the MLE of the normal distribution with unknown mean and variance.
Multivariate circular observations, i.e. points on a torus are nowadays very common. Multivariate wrapped models are often appropriate to describe data points scattered on p-dimensional torus. However, statistical inference based on this model is quite complicated since each contribution in the log likelihood involve an infinite sum of indices in Z^p where p is the dimension of the problem. To overcome this, two estimates procedures based on Expectation Maximization and Classification Expectation Maximization algorithms are proposed that worked well in moderate dimension size. The performance …
The problem of characterizing a multivariate distribution of a random vector using examination of univariate combinations of vector components is an essential issue of multivariate analysis. The likelihood principle plays a prominent role in developing powerful statistical inference tools. In this context, we raise the question: can the univariate likelihood function based on a random vector be used to provide the uniqueness in reconstructing the vector distribution? In multivariate normal (MN) frameworks, this question links to a reverse of Cochran's theorem that concerns the distribution of …
Multivariate probit models (MPM) have the appealing feature of capturing some of the dependence structure between the components of multidimensional binary responses. The key for the dependence modelling is the covariance matrix of an underlying latent multivariate Gaussian. Most approaches to MLE in multivariate probit regression rely on MCEM algorithms to avoid computationally intensive evaluations of multivariate normal orthant probabilities. As an alternative to the much used Gibbs sampler a new SMC sampler for truncated multivariate normals is proposed. The algorithm proceeds in two stage…
This article describes a method for using optimization to derive efficient independent transition functions for Markov chain Monte Carlo simulations. Our interest is in sampling from a posterior density $π(x)$ for problems in which the dimension of the model space is large, $π(x)$ is multimodal with regions of low probability separating the modes, and evaluation of the likelihood is expensive. We restrict our attention to the special case for which the target density is the product of a multivariate Gaussian prior and a likelihood function for which the errors in observations are additive and …
We obtain explicit $p$-Wasserstein distance error bounds between the distribution of the multi-parameter MLE and the multivariate normal distribution. Our general bounds are given for possibly high-dimensional, independent and identically distributed random vectors. Our general bounds are of the optimal $\mathcal{O}(n^{-1/2})$ order. Explicit numerical constants are given when $p\in(1,2]$, and in the case $p>2$ the bounds are explicit up to a constant factor that only depends on $p$. We apply our general bounds to derive Wasserstein distance error bounds for the multivariate normal approximati…
Multivariate normal mixtures provide a flexible model for high-dimensional data. They are widely used in statistical genetics, statistical finance, and other disciplines. Due to the unboundedness of the likelihood function, classical likelihood-based methods, which may have nice practical properties, are inconsistent. In this paper, we recommend a penalized likelihood method for estimating the mixing distribution. We show that the maximum penalized likelihood estimator is strongly consistent when the number of components has a known upper bound. We also explore a convenient EM-algorithm for co…
There has been a growing interest in providing models for multivariate spatial processes. A majority of these models specify a parametric matrix covariance function. Based on observations, the parameters are estimated by maximum likelihood or variants thereof. While the asymptotic properties of maximum likelihood estimators for univariate spatial processes have been analyzed in detail, maximum likelihood estimators for multivariate spatial processes have not received their deserved attention yet. In this article we consider the classical increasing-domain asymptotic setting restricting the min…
The normalized maximum likelihood (NML) is one of the most important distribution in coding theory and statistics. NML is the unique solution (if exists) to the pointwise minimax regret problem. However, NML is not defined even for simple family of distributions such as the normal distributions. Since there does not exist any meaningful minimax-regret distribution for such case, it is pointed out that NML with luckiness (LNML) can be employed as an alternative to NML. In this paper, we develop the closed form of LNMLs for multivariate normal distributions.
We provide a closed-form estimator based on the VARMA representation for the unrestricted multivariate GARCH(1,1). We show that all parameters can be derived using basic linear algebra tools. We show that the estimator is consistent and asymptotically normal distributed. Our results allow also to derive a closed form for the parameters in the context of temporal aggregation of multivariate GARCH(1,1) by solving the equations as in Hafner [2008].
We propose new model selection criteria based on generalized ridge estimators dominating the maximum likelihood estimator under the squared risk and the Kullback-Leibler risk in multivariate linear regression. Our model selection criteria have the following favorite properties: consistency, unbiasedness, uniformly minimum variance. Consistency is proven under an asymptotic structure $\frac{p}{n}\to c$ where $n$ is the sample size and $p$ is the parameter dimension of the response variables. In particular, our proposed class of estimators dominates the maximum likelihood estimator under the squ…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
The problem of reducing the bias of maximum likelihood estimator in a general multivariate elliptical regression model is considered. The model is very flexible and allows the mean vector and the dispersion matrix to have parameters in common. Many frequently used models are special cases of this general formulation, namely: errors-in-variables models, nonlinear mixed-effects models, heteroscedastic nonlinear models, among others. In any of these models, the vector of the errors may have any multivariate elliptical distribution. We obtain the second-order bias of the maximum likelihood estimat…
This paper investigates improved testing inferences under a general multivariate elliptical regression model. The model is very flexible in terms of the specification of the mean vector and the dispersion matrix, and of the choice of the error distribution. The error terms are allowed to follow a multivariate distribution in the class of the elliptical distributions, which has the multivariate normal and Student-t distributions as special cases. We obtain Skovgaard's adjusted likelihood ratio statistics and Barndorff-Nielsen's adjusted signed likelihood ratio statistics and we conduct a simula…
We consider the change-point detection in multivariate continuous and integer valued time series. We propose a Wald-type statistic based on the estimator performed by a general contrast function; which can be constructed from the likelihood, a quasi-likelihood, a least squares method, etc. Sufficient conditions are provided to ensure that the statistic convergences to a well-known distribution under the null hypothesis (of no change) and diverges to infinity under the alternative; which establishes the consistency of the procedure. Some examples are detailed to illustrate the scope of applicat…
This paper is devoted to the multivariate estimation of a vector of Poisson means. A novel loss function that penalises bad estimates of each of the parameters and the sum (or equivalently the mean) of the parameters is introduced. Under this loss function, a class of minimax estimators that uniformly dominate the maximum likelihood estimator is derived. Crucially, these methods have the property that for estimating a given component parameter, the full data vector is utilised. Estimators in this class can be fine-tuned to limit shrinkage away from the maximum likelihood estimator, thereby avo…
We consider quasi maximum likelihood (QML) estimation for general non-Gaussian discrete-ime linear state space models and equidistantly observed multivariate Lévy-driven continuoustime autoregressive moving average (MCARMA) processes. In the discrete-time setting, we prove strong consistency and asymptotic normality of the QML estimator under standard moment assumptions and a strong-mixing condition on the output process of the state space model. In the second part of the paper, we investigate probabilistic and analytical properties of equidistantly sampled continuous-time state space models a…
Often of primary interest in the analysis of multivariate data are the copula parameters describing the dependence among the variables, rather than the univariate marginal distributions. Since the ranks of a multivariate dataset are invariant to changes in the univariate marginal distributions, rank-based estimators are natural candidates for semiparametric copula estimation. Asymptotic information bounds for such estimators can be obtained from an asymptotic analysis of the rank likelihood, that is, the probability of the multivariate ranks. In this article, we obtain limiting normal distribu…
Statistical estimation and inference for marginal hazard models with varying coefficients for multivariate failure time data are important subjects in survival analysis. A local pseudo-partial likelihood procedure is proposed for estimating the unknown coefficient functions. A weighted average estimator is also proposed in an attempt to improve the efficiency of the estimator. The consistency and asymptotic normality of the proposed estimators are established and standard error formulas for the estimated coefficients are derived and empirically tested. To reduce the computational burden of the…
We establish global rates of convergence of the Maximum Likelihood Estimator (MLE) of a multivariate distribution function in the case of (one type of) "interval censored" data. The main finding is that the rate of convergence of the MLE in the Hellinger metric is no worse than $n^{-1/3} (\log n)^γ$ for $γ= (5d - 4)/6$.
Finite mixtures of multivariate normal distributions have been widely used in empirical applications in diverse fields such as statistical genetics and statistical finance. Testing the number of components in multivariate normal mixture models is a long-standing challenge even in the most important case of testing homogeneity. This paper develops likelihood-based tests of the null hypothesis of $M_0$ components against the alternative hypothesis of $M_0 + 1$ components for a general $M_0 \geq 1$. For heteroscedastic normal mixtures, we propose an EM test and derive the asymptotic distribution …
Nonparametric tests for equality of multivariate distributions are frequently desired in research. It is commonly required that test-procedures based on relatively small samples of vectors accurately control the corresponding Type I Error (TIE) rates. Often, in the multivariate testing, extensions of null-distribution-free univariate methods, e.g., Kolmogorov-Smirnov and Cramer-von Mises type schemes, are not exact, since their null distributions depend on underlying data distributions. The present paper extends the density-based empirical likelihood technique in order to nonparametrically app…
We consider estimation of the covariance matrix of a multivariate random vector under the constraint that certain covariances are zero. We first present an algorithm, which we call Iterative Conditional Fitting, for computing the maximum likelihood estimator of the constrained covariance matrix, under the assumption of multivariate normality. In contrast to previous approaches, this algorithm has guaranteed convergence properties. Dropping the assumption of multivariate normality, we show how to estimate the covariance matrix in an empirical likelihood approach. These approaches are then compa…
We study the adaptation properties of the multivariate log-concave maximum likelihood estimator over three subclasses of log-concave densities. The first consists of densities with polyhedral support whose logarithms are piecewise affine. The complexity of such densities~$f$ can be measured in terms of the sum $Γ(f)$ of the numbers of facets of the subdomains in the polyhedral subdivision of the support induced by $f$. Given $n$ independent observations from a $d$-dimensional log-concave density with $d \in \{2,3\}$, we prove a sharp oracle inequality, which in particular implies that the Kull…
Multivariate linear regressions are widely used statistical tools in many applications to model the associations between multiple related responses and a set of predictors. To infer such associations, it is often of interest to test the structure of the regression coefficients matrix, and the likelihood ratio test (LRT) is one of the most popular approaches in practice. Despite its popularity, it is known that the classical $χ^2$ approximations for LRTs often fail in high-dimensional settings, where the dimensions of responses and predictors $(m,p)$ are allowed to grow with the sample size $n$…
We propose a new testing procedure about the tail weight parameter of multivariate Student $t$ distributions by having recourse to the Le Cam methodology. Our test is asymptotically as efficient as the classical likelihood ratio test, but outperforms the latter by its flexibility and simplicity: indeed, our approach allows to estimate the location and scatter nuisance parameters by any root-$n$ consistent estimators, hereby avoiding numerically complex maximum likelihood estimation. The finite-sample properties of our test are analyzed in a Monte Carlo simulation study, and we apply our method…
This paper considers a multivariate spatial random field, with each component having univariate marginal distributions of the skew-Gaussian type. We assume that the field is defined spatially on the unit sphere embedded in $\mathbb{R}^3$, allowing for modeling data available over large portions of planet Earth. This model admits explicit expressions for the marginal and cross covariances. However, the $n$-dimensional distributions of the field are difficult to evaluate, because it requires the sum of $2^n$ terms involving the cumulative and probability density functions of a $n$-dimensional Ga…
Consider a multivariate Lévy-driven Ornstein-Uhlenbeck process where the stationary distribution or background driving Lévy process is from a parametric family. We derive the likelihood function assuming that the innovation term is absolutely continuous. Two examples are studied in detail: the process where the stationary distribution or background driving Lévy process is given by a weak variance alpha-gamma process, which is a multivariate generalisation of the variance gamma process created using weak subordination. In the former case, we give an explicit representation of the background dri…
We study a non-parametric approach to multivariate density estimation. The estimators are piecewise constant density functions supported by binary partitions. The partition of the sample space is learned by maximizing the likelihood of the corresponding histogram on that partition. We analyze the convergence rate of the sieve maximum likelihood estimator, and reach a conclusion that for a relatively rich class of density functions the rate does not directly depend on the dimension. This suggests that, under certain conditions, this method is immune to the curse of dimensionality, in the sense …
In this work, the estimation of the multivariate normal mean by different classes of shrinkage estimators is investigated. The risk associated with the balanced loss function is used to compare two estimators. We start by considering estimators that generalize the James-Stein estimator and show that these estimators dominate the maximum likelihood estimator (MLE), therefore are minimax, when the shrinkage function satisfies some conditions. Then, we treat estimators of polynomial form and prove the increase of the degree of the polynomial allows us to build a better estimator from the one prev…
The Ohio State University（俄亥俄州立大学）；University of Southern California（南加州大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
We study the problem of learning multivariate log-concave densities with respect to a global loss function. We obtain the first upper bound on the sample complexity of the maximum likelihood estimator (MLE) for a log-concave density on $\mathbb{R}^d$, for all $d \geq 4$. Prior to this work, no finite sample upper bound was known for this estimator in more than $3$ dimensions. In more detail, we prove that for any $d \geq 1$ and $ε>0$, given $\tilde{O}_d((1/ε)^{(d+3)/2})$ samples drawn from an unknown log-concave density $f_0$ on $\mathbb{R}^d$, the MLE outputs a hypothesis $h$ that with high p…
A weighted likelihood technique for robust estimation of a multivariate Wrapped Normal distribution for data points scattered on a p-dimensional torus is proposed. The occurrence of outliers in the sample at hand can badly compromise inference for standard techniques such as maximum likelihood method. Therefore, there is the need to handle such model inadequacies in the fitting process by a robust technique and an effective down-weighting of observations not following the assumed model. Furthermore, the employ of a robust method could help in situations of hidden and unexpected substructures i…
