Diffusion processes are widely used for modelling real-world phenomena. Except for select cases however, analytical expressions do not exist for a diffusion process' transitional probabilities. It is proposed that the cumulant truncation procedure can be applied to predict the evolution of the cumulants of the system. These predictions may be subsequently used within the saddlepoint procedure to approximate the transitional probabilities. An approximation to the likelihood of the diffusion system is then easily derived. The method is applicable for a wide-range of diffusion systems - including multivariate, irreducible diffusion systems that existing estimation schemes struggle with. Not only is the accuracy of the saddlepoint comparable with the Hermite expansion - a popular approximation to a diffusion system's transitional density - it also appears to be less susceptible to increasing lags between successive samplings of the diffusion process. Furthermore, the saddlepoint is more stable in regions of the parameter space that are far from the maximum likelihood estimates. Hence, the saddlepoint method can be naturally incorporated within a Markov Chain Monte Carlo (MCMC) routine in order to provide reliable estimates and credibility intervals of the diffusion model's parameters. The method is applied to fit the Heston model to daily observations of the S&P 500 and VIX indices from December 2009 to November 2010.
This paper provides closed-form expansions for the log-likelihood function of multivariate diffusions sampled at discrete time intervals. The coefficients of the expansion are calculated explicitly by exploiting the special structure afforded by the diffusion model. Examples of interest in financial statistics and Monte Carlo evidence are included, along with the convergence of the expansion to the true likelihood function.
In this paper, we begin our discussion with some of the well-known methods available in the literature for the estimation of the parameters of a univariate/multivariate stable distribution. Based on the available methods, a new hybrid method is proposed for the estimation of the parameters of a univariate stable distribution. The proposed method is further used for the estimation of the parameters of a strictly multivariate stable distribution. The efficiency, accuracy, and simplicity of the new method is shown through Monte-Carlo simulation. Finally, we apply the proposed method to the univar…
The distributional transform (DT) is amongst the computational methods used for estimation of high-dimensional multivariate normal copula models with discrete responses. Its advantage is that the likelihood can be derived conveniently under the theory for copula models with continuous margins, but there has not been a clear analysis of the adequacy of this method. We investigate the small-sample and asymptotic efficiency of the method for estimating high-dimensional multivariate normal copula models with univariate Bernoulli, Poisson, and negative binomial margins, and show that the DT approxi…
Multivariate circular observations, i.e. points on a torus are nowadays very common. Multivariate wrapped models are often appropriate to describe data points scattered on p-dimensional torus. However, statistical inference based on this model is quite complicated since each contribution in the log likelihood involve an infinite sum of indices in Z^p where p is the dimension of the problem. To overcome this, two estimates procedures based on Expectation Maximization and Classification Expectation Maximization algorithms are proposed that worked well in moderate dimension size. The performance …
The t-distribution has many useful applications in robust statistical analysis. The parameter estimation of the t-distribution is carried out using ML estimation method, and the ML estimates are obtained via the EM algorithm. In this study, we consider an alternative estimation method for all the parameters of the multivariate-t distribution using the MLq estimation method. We adapt the EM algorithm to obtain the MLq estimates for all the parameters. We provide a small simulation study to illustrate the performance of the MLq estimators over the ML estimators and observe that the MLq estimator…
We investigate a method for extracting nonlinear principal components (NPCs). These NPCs maximize variation subject to smoothness and orthogonality constraints; but we allow for a general class of constraints and multivariate probability densities, including densities without compact support and even densities with algebraic tails. We provide primitive sufficient conditions for the existence of these NPCs. By exploiting the theory of continuous-time, reversible Markov diffusion processes, we give a different interpretation of these NPCs and the smoothness constraints. When the diffusion matrix…
Multivariate density estimation is a popular technique in statistics with wide applications including regression models allowing for heteroskedasticity in conditional variances. The estimation problems become more challenging when observations are missing in one or more variables of the multivariate vector. A flexible class of mixture of tensor products of kernel densities is proposed which allows for easy implementation of imputation methods using Gibbs sampling and shown to have superior performance compared to some of the exisiting imputation methods currently available in literature. Numer…
A novel approach to obtain weighted likelihood estimates of multivariate location and scatter is discussed. A weighting scheme is proposed that is based on the distribution of the Mahalanobis distances rather than the distribution of the data at the assumed model. This strategy allows to avoid the curse of dimensionality affecting non-parametric density estimation, that is involved in the construction of the weights through the Pearson residuals Markatou et al (1998). Then, weighted likelihood based outlier detection rules and robust dimensionality reduction techniques are developed. The effec…
We consider regularly varying random vectors. Our goal is to estimate in a non-parametric way some characteristics related to conditioning on an extreme event, like the tail dependence coefficient. We introduce a quasi-spectral decomposition that allow to improve efficiency of estimators. Asymptotic normality of estimators is based on weak convergence of tail empirical processes. Theoretical results are supported by simulation studies.
For multivariate time series driven by underlying states, hidden Markov models (HMMs) constitute a powerful framework which can be flexibly tailored to the situation at hand. However, in practice it can be challenging to choose an adequate emission distribution for multivariate observation vectors. For example, the marginal data distribution may not immediately reveal the within-state distributional form, and also the different data streams may operate on different supports, rendering the common approach of using a multivariate normal distribution inadequate. Here we explore a nonparametric es…
Many applications in risk analysis, especially in environmental sciences, require the estimation of the dependence among multivariate maxima. A way to do this is by inferring the Pickands dependence function of the underlying extreme-value copula. A nonparametric estimator is constructed as the sample equivalent of a multivariate extension of the madogram. Shape constraints on the family of Pickands dependence functions are taken into account by means of a representation in terms of a specific type of Bernstein polynomials. The large-sample theory of the estimator is developed and its finite-s…
Multivariate Poisson intensity estimation via low-rank tensor decomposition
Auburn University（奥本大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；University of California, Los Angeles（加州大学洛杉矶分校）；University of California, San Diego（加州大学圣迭戈分校）
It is often assumed that events cannot occur simultaneously when modelling data with point processes. This raises a problem as real-world data often contains synchronous observations due to aggregation or rounding, resulting from limitations on recording capabilities and the expense of storing high volumes of precise data. In order to gain a better understanding of the relationships between processes, we consider modelling the aggregated event data using multivariate Hawkes processes, which offer a description of mutually-exciting behaviour and have found wide applications in areas including s…
The Fay-Herriot model is a standard model for direct survey estimators in which the true quantity of interest, the superpopulation mean, is latent and its estimation is improved through the use of auxiliary covariates. In the context of small area estimation, these estimates can be further improved by borrowing strength across spatial region or by considering multiple outcomes simultaneously. We provide here two formulations to perform small area estimation with Fay-Herriot models that include both multivariate outcomes and latent spatial dependence. We consider two model formulations, one in …
Nonparametric estimators of a regression function with circular response and Rd-valued predictor are considered in this work. Local polynomial type estimators are proposed and studied. Expressions for their asymptotic biases and variances are derived, and some guidelines to select asymptotically local optimal bandwidth matrices are also given. The finite sample behavior of the proposed estimators is assessed through simulations and their performance is also illustrated with a real data set.
Multivariate sign functions are often used for robust estimation and inference. We propose using data dependent weights in association with such functions. The proposed weighted sign functions retain desirable robustness properties, while significantly improving efficiency in estimation and inference compared to unweighted multivariate sign-based methods. Using weighted signs, we demonstrate methods of robust location estimation and robust principal component analysis. We extend the scope of using robust multivariate methods to include robust sufficient dimension reduction and functional outli…
In the field of finance, insurance, and system reliability, etc., it is often of interest to measure the dependence among variables by modeling a multivariate distribution using a copula. The copula models with parametric assumptions are easy to estimate but can be highly biased when such assumptions are false, while the empirical copulas are non-smooth and often not genuine copula making the inference about dependence challenging in practice. As a compromise, the empirical Bernstein copula provides a smooth estimator but the estimation of tuning parameters remains elusive. In this paper, by u…
This paper proposes a maximum-likelihood approach to jointly estimate marginal conditional quantiles of multivariate response variables in a linear regression framework. We consider a slight reparameterization of the Multivariate Asymmetric Laplace distribution proposed by Kotz et al (2001) and exploit its location-scale mixture representation to implement a new EM algorithm for estimating model parameters. The idea is to extend the link between the Asymmetric Laplace distribution and the well-known univariate quantile regression model to a multivariate context, i.e. when a multivariate depend…
Statistical inference for highly multivariate point pattern data is challenging due to complex models with large numbers of parameters. In this paper, we develop numerically stable and efficient parameter estimation and model selection algorithms for a class of multivariate log Gaussian Cox processes. The methodology is applied to a highly multivariate point pattern data set from tropical rain forest ecology.
The Hodrick-Prescott filter represents one of the most popular method for trend-cycle extraction in macroeconomic time series. In this paper we provide a multivariate generalization of the Hodrick-Prescott filter, based on the seemingly unrelated time series approach. We first derive closed-form expressions linking the signal-noise matrix ratio to the parameters of the VARMA representation of the model. We then show that the parameters can be estimated using a recently introduced method, called "Moment Estimation Through Aggregation (META)". This method replaces the traditional multivariate li…
Non-parametric estimation of a multivariate density estimation is tackled via a method which combines traditional local smoothing with a form of global smoothing but without imposing a rigid structure. Simulation work delivers encouraging indications on the effectiveness of the method. An application to density-based clustering illustrates a possible usage.
This paper presents a practical and simple fully nonparametric multivariate smoothing procedure that adapts to the underlying smoothness of the true regression function. Our estimator is easily computed by successive application of existing base smoothers (without the need of selecting an optimal smoothing parameter), such as thin-plate spline or kernel smoothers. The resulting smoother has better out of sample predictive capabilities than the underlying base smoother, or competing structurally constrained models (GAM) for small dimension (3 < d < 8) and moderate sample size (n < 800). Moreove…
We consider covariance estimation in the multivariate generalized Gaussian distribution (MGGD) and elliptically symmetric (ES) distribution. The maximum likelihood optimization associated with this problem is non-convex, yet it has been proved that its global solution can be often computed via simple fixed point iterations. Our first contribution is a new analysis of this likelihood based on geodesic convexity that requires weaker assumptions. Our second contribution is a generalized framework for structured covariance estimation under sparsity constraints. We show that the optimizations can b…
Based on the concept of a Lévy copula to describe the dependence structure of a multivariate Lévy process we present a new estimation procedure. We consider a parametric model for the marginal Lévy processes as well as for the Lévy copula and estimate the parameters by a two-step procedure. We first estimate the parameters of the marginal processes, and then estimate in a second step only the dependence structure parameter. For infinite Lévy measures we truncate the small jumps and base our statistical analysis on the large jumps of the model. Prominent example will be a bivariate stable \lp, …
Extreme-value copulas arise in the asymptotic theory for componentwise maxima of independent random samples. An extreme-value copula is determined by its Pickands dependence function, which is a function on the unit simplex subject to certain shape constraints that arise from an integral transform of an underlying measure called spectral measure. Multivariate extensions are provided of certain rank-based nonparametric estimators of the Pickands dependence function. The shape constraint that the estimator should itself be a Pickands dependence function is enforced by replacing an initial estima…
The multivariate probit is popular for modeling correlated binary data, with an attractive balance of flexibility and simplicity. However, considerable challenges remain in computation and in devising a clear statistical framework. Interest in the multivariate probit has increased in recent years. Current applications include genomics and precision medicine, where simultaneous modeling of multiple traits may be of interest, and computational efficiency is an important consideration. We propose a fast method for multivariate probit estimation via a two-stage composite likelihood. We explore com…
Covariance estimation is essential yet underdeveloped for analyzing multivariate functional data. We propose a fast covariance estimation method for multivariate sparse functional data using bivariate penalized splines. The tensor-product B-spline formulation of the proposed method enables a simple spectral decomposition of the associated covariance operator and explicit expressions of the resulting eigenfunctions as linear combinations of B-spline bases, thereby dramatically facilitating subsequent principal component analysis. We derive a fast algorithm for selecting the smoothing parameters…
We propose simple methods for multivariate diffusion bridge simulation, which plays a fundamental role in simulation-based likelihood and Bayesian inference for stochastic differential equations. By a novel application of classical coupling methods, the new approach generalizes a previously proposed simulation method for one-dimensional bridges to the multi-variate setting. First a method of simulating approximate, but often very accurate, diffusion bridges is proposed. These approximate bridges are used as proposal for easily implementable MCMC algorithms that produce exact diffusion bridges.…
We study a group lasso estimator for the multivariate linear regression model that accounts for correlated error terms. A block coordinate descent algorithm is used to compute this estimator. We perform a simulation study with categorical data and multivariate time series data, typical settings with a natural grouping among the predictor variables. Our simulation studies show the good performance of the proposed group lasso estimator compared to alternative estimators. We illustrate the method on a time series data set of gene expressions.
The prevalence of spatially referenced multivariate data has impelled researchers to develop a procedure for the joint modeling of multiple spatial processes. This ordinarily involves modeling marginal and cross-process dependence for any arbitrary pair of locations using a multivariate spatial covariance function. However, building a flexible multivariate spatial covariance function that is nonnegative definite is challenging. Here, we propose a semiparametric approach for multivariate spatial covariance function estimation with approximate Matérn marginals and highly flexible cross-covarianc…
Modeling data with multivariate count responses is a challenging problem due to the discrete nature of the responses. Existing methods for univariate count responses cannot be easily extended to the multivariate case since the dependency among multiple responses needs to be properly accommodated. In this paper, we propose a multivariate Poisson log-normal regression model for multivariate data with count responses. By simultaneously estimating the regression coefficients and inverse covariance matrix over the latent variables with an efficient Monte Carlo EM algorithm, the proposed regression …
In this paper we develop a Bayesian procedure for estimating multivariate stochastic volatility (MSV) using state space models. A multiplicative model based on inverted Wishart and multivariate singular beta distributions is proposed for the evolution of the volatility, and a flexible sequential volatility updating is employed. Being computationally fast, the resulting estimation procedure is particularly suitable for on-line forecasting. Three performance measures are discussed in the context of model selection: the log-likelihood criterion, the mean of standardized one-step forecast errors, …
Causal inference is widely used in various fields, such as biology, psychology and economics, etc. In observational studies, we need to balance the covariates before estimating causal effect. This study extends the one-dimensional entropy balancing method to multiple dimensions to balance the covariates. Both parametric and nonparametric methods are proposed to estimate the causal effect of multivariate continuous treatments and theoretical properties of the two estimations are provided. Furthermore, the simulation results show that the proposed method is better than other methods in various c…
In data science, vector autoregression (VAR) models are popular in modeling multivariate time series in the environmental sciences and other applications. However, these models are computationally complex with the number of parameters scaling quadratically with the number of time series. In this work, we propose a so-called neighborhood vector autoregression (NVAR) model to efficiently analyze large-dimensional multivariate time series. We assume that the time series have underlying neighborhood relationships, e.g., spatial or network, among them based on the inherent setting of the problem. W…
Multivariate meta-analysis can be adapted to a wide range of situations for multiple outcomes and multiple treatment groups when combining studies together. The within-study correlation between effect sizes is often assumed known in multivariate meta-analysis while it is not always known practically. In this paper, we propose a generic method to approximate the within-study covariance for effect sizes in multivariate meta-analysis and apply this method to the scenarios with multiple outcomes and one outcome with multiple treatment groups respectively.
A new multivariate stochastic volatility estimation procedure for financial time series is proposed. A Wishart autoregressive process is considered for the volatility precision covariance matrix, for the estimation of which a two step procedure is adopted. The first step is the conditional inference on the autoregressive parameters and the second step is the unconditional inference, based on a Newton-Raphson iterative algorithm. The proposed methodology, which is mostly Bayesian, is suitable for medium dimensional data and it bridges the gap between closed-form estimation and simulation-based …
We propose a non-parametric statistical procedure for detecting multiple change-points in multidimensional signals. The method is based on a test statistic that generalizes the well-known Kruskal-Wallis procedure to the multivariate setting. The proposed approach does not require any knowledge about the distribution of the observations and is parameter-free. It is computationally efficient thanks to the use of dynamic programming and can also be applied when the number of change-points is unknown. The method is shown through simulations to be more robust than alternatives, particularly when fa…
For the multivariate COGARCH process, we obtain explicit expressions for the second-order structure of the "squared returns" process observed on an equidistant grid. Based on this, we present a generalized method of moments estimator for its parameters. Under appropriate moment and strong mixing conditions, we show that the resulting estimator is consistent and asymptotically normal. Sufficient conditions for strong mixing, stationarity and identifiability of the model parameters are discussed in detail. We investigate the finite sample behavior of the estimator in a simulation study.
In multivariate time series, the estimation of the covariance matrix of the observation innovations plays an important role in forecasting as it enables the computation of the standardized forecast error vectors as well as it enables the computation of confidence bounds of the forecasts. We develop an on-line, non-iterative Bayesian algorithm for estimation and forecasting. It is empirically found that, for a range of simulated time series, the proposed covariance estimator has good performance converging to the true values of the unknown observation covariance matrix. Over a simulated time se…
In this paper, a nonparametric estimator is proposed for estimating the L1-median for multivariate conditional distribution when the covariates take values in an infinite dimensional space. The multivariate case is more appropriate to predict the components of a vector of random variables simultaneously rather than predicting each of them separately. While estimating the conditional L1-median function using the well-known Nadarya-Waston estimator, we establish the strong consistency of this estimator as well as the asymptotic normality. We also present some simulations and provide how to built…
In this paper, we study convex optimization methods for computing the trace norm regularized least squares estimate in multivariate linear regression. The so-called factor estimation and selection (FES) method, recently proposed by Yuan et al. [22], conducts parameter estimation and factor selection simultaneously and have been shown to enjoy nice properties in both large and finite samples. To compute the estimates, however, can be very challenging in practice because of the high dimensionality and the trace norm constraint. In this paper, we explore a variant of Nesterov's smooth method [20]…
We propose a new method for multivariate response regression and covariance estimation when elements of the response vector are of mixed types, for example some continuous and some discrete. Our method is based on a model which assumes the observable mixed-type response vector is connected to a latent multivariate normal response linear regression through a link function. We explore the properties of this model and show its parameters are identifiable under reasonable conditions. We impose no parametric restrictions on the covariance of the latent normal other than positive definiteness, there…
Local-Polynomial Estimation for Multivariate Regression Discontinuity Designs
Institute of Economic Research Hitotsubashi University（一桥大学经济研究所）；Graduate School of Economics and Management, Tohoku University（东北大学经济管理学院）；Center for Spatial Information Science, The University of Tokyo（东京大学空间信息科学中心）
Convex Reparameterization and Self-Concordant Algorithms for Multivariate Regression with Covariance Estimation
School of Statistics, University of Minnesota Twin Cities（明尼苏达大学双城分校统计学院）；Department of Applied Mathematics & Statistics, Stony Brook University（石溪大学应用数学与统计系）
In many studies multivariate event time data are generated from clusters having a possibly complex association pattern. Flexible models are needed to capture this dependence. Vine copulas serve this purpose. Inference methods for vine copulas are available for complete data. Event time data, however, are often subject to right-censoring. As a consequence, the existing inferential tools, e.g. likelihood estimation, need to be adapted. A two-stage estimation approach is proposed. First, the marginal distributions are modeled. Second, the dependence structure modeled by a vine copula is estimated…
