We study asymptotic behavior of maximum likelihood estimator for a time inhomogeneous diffusion process given by a SDE $dX_t=αb(t)X_t dt + σ(t) dB_t$, $t\in[0,T)$, with a parameter $α\in R$, where $T\in(0,\infty]$ and $(B_t)_{t\in[0,T)}$ is a standard Wiener process. We formulate sufficient conditions under which the MLE of $α$ normalized by Fisher information converges to the limit distribution of Dickey-Fuller statistics. Next we study a SDE $dY_t=αb(t)a(Y_t) dt + σ(t) dB_t$, $t\in[0,T)$, with a perturbed drift satisfying $a(x)=x+O(1+|x|^γ)$ with some $γ\in[0,1)$. We give again sufficient conditions under which the MLE of $α$ normalized by Fisher information converges to the limit distribution of Dickey-Fuller statistics.
We have shown in previous work that statistical inference for cooperative sequential adsorption model can be based on maximum likelihood estimation. In this paper we continue this research and establish asymptotic normality of the maximum likelihood estimator in thermodynamic limit. We also perform and discuss some numerical simulations of the model.
In finite mixtures of location-scale distributions, if there is no constraint or penalty on the parameters, then the maximum likelihood estimator does not exist because the likelihood is unbounded. To avoid this problem, we consider a penalized likelihood, where the penalty is a function of the minimum of the ratios of the scale parameters and the sample size. It is shown that the penalized maximum likelihood estimator is strongly consistent. We also analyze the consistency of a penalized maximum likelihood estimator where the penalty is imposed on the scale parameters themselves.
We consider statistical models driven by Gaussian and non-Gaussian self-similar processes with long memory and we construct maximum likelihood estimators (MLE) for the drift parameter. Our approach is based on the approximation by random walks of the driving noise. We study the asymptotic behavior of the estimators and we give some numerical simulations to illustrate our results.
In this paper, we study the asymptotic normality of the conditional maximum likelihood (ML) estimators for the truncated regression model and the Tobit model. We show that under the general setting assumed in his book, the conjectures made by Hayashi (2000) \footnote{see page 516, and page 520 of Hayashi (2000).} about the asymptotic normality of the conditional ML estimators for both models are true, namely, a sufficient condition is the nonsingularity of $\mathbf{x_tx'_t}$.
We consider parameter estimation in finite hidden state space Markov models with time-dependent inhomogeneous noise, where the inhomogeneity vanishes sufficiently fast. Based on the concept of asymptotic mean stationary processes we prove that the maximum likelihood and a quasi-maximum likelihood estimator (QMLE) are strongly consistent. The computation of the QMLE ignores the inhomogeneity, hence, is much simpler and robust. The theory is motivated by an example from biophysics and applied to a Poisson- and linear Gaussian model.
The paper studies large sample asymptotic properties of the Maximum Likelihood Estimator (MLE) for the parameter of a continuous time Markov chain, observed in white noise. Using the method of weak convergence of likelihoods due to I.Ibragimov and R.Khasminskii, consistency, asymptotic normality and convergence of moments are established for MLE under certain strong ergodicity conditions of the chain.
Composite likelihoods are a class of alternatives to the full likelihood which are widely used in many situations in which the likelihood itself is intractable. A composite likelihood may be computed without the need to specify the full distribution of the response, which means that in some situations the resulting estimator will be more robust to model misspecification than the maximum likelihood estimator. The purpose of this note is to show that such increased robustness is not guaranteed. An example is given in which various marginal composite likelihood estimators are inconsistent under m…
We study the maximum likelihood estimator of the drift parameters of a stochastic differential equation, with both drift and diffusion coefficients constant on the positive and negative axis, yet discontinuous at zero. This threshold diffusion is called drifted Oscillating Brownian motion.For this continuously observed diffusion, the maximum likelihood estimator coincide with a quasi-likelihood estimator with constant diffusion term. We show that this estimator is the limit, as observations become dense in time, of the (quasi)-maximum likelihood estimator based on discrete observations. In lon…
This paper deals with the problems of consistence and strong consistence of the maximum likelihood estimators of the mean and variance of the drift fractional Brownian motions observed at discrete time instants. A central limit theorem for these estimators is also obtained by using the Malliavin calculus.
A parameter estimation problem is considered for a diagonaliazable stochastic evolution equation using a finite number of the Fourier coefficients of the solution. The equation is driven by additive noise that is white in space and fractional in time with the Hurst parameter $H\geq 1/2$. The objective is to study asymptotic properties of the maximum likelihood estimator as the number of the Fourier coefficients increases. A necessary and sufficient condition for consistency and asymptotic normality is presented in terms of the eigenvalues of the operators in the equation.
Indirect inference estimators (i.e., simulation-based minimum distance estimators) in a parametric model that are based on auxiliary non-parametric maximum likelihood density estimators are shown to be asymptotically normal. If the parametric model is correctly specified, it is furthermore shown that the asymptotic variance-covariance matrix equals the inverse of the Fisher-information matrix. These results are based on uniform-in-parameters convergence rates and a uniform-in-parameters Donsker-type theorem for non-parametric maximum likelihood density estimators.
The use of {\it Mathematica} in deriving mean likelihood estimators is discussed. Comparisons between the maximum likelihood estimator, the mean likelihood estimator and the Bayes estimate based on a Jeffrey's noninformative prior using the criteria mean-square error and Pitman measure of closeness. Based on these criteria we find that for the first-order moving-average time series model, the mean likelihood estimator outperforms the maximum likelihood estimator and the Bayes estimator with a Jeffrey's noninformative prior. {\it Mathematica} was used for symbolic and numeric computations as we…
Upper and lower class functions for the maximum likelihood estimator of the arrival and the service rates in a $GI/G/1$ queue are studied and the results are verified for M/M/1 queue.
While the asymptotic normality of the maximum likelihood estimator under regularity conditions is long established, this paper derives explicit bounds for the bounded Wasserstein distance between the distribution of the maximum likelihood estimator (MLE) and the normal distribution. For this task, we employ Stein's method. We focus on independent and identically distributed random variables, covering both discrete and continuous distributions as well as exponential and non-exponential families. In particular, a closed form expression of the MLE is not required. We also use a perturbation metho…
We consider covariance parameter estimation for a Gaussian process under inequality constraints (boundedness, monotonicity or convexity) in fixed-domain asymptotics. We address the estimation of the variance parameter and the estimation of the microergodic parameter of the Matérn and Wendland covariance functions. First, we show that the (unconstrained) maximum likelihood estimator has the same asymptotic distribution, unconditionally and conditionally to the fact that the Gaussian process satisfies the inequality constraints. Then, we study the recently suggested constrained maximum likelihoo…
Maximum likelihood estimation has been extensively used in the joint analysis of repeated measurements and survival time. However, there is a lack of theoretical justification of the asymptotic properties for the maximum likelihood estimators. This paper intends to fill this gap. Specifically, we prove the consistency of the maximum likelihood estimators and derive their asymptotic distributions. The maximum likelihood estimators are shown to be semiparametrically efficient.
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
We give a thorough description of the asymptotic property of the maximum likelihood estimator (MLE) of the skewness parameter of a Skew Brownian Motion (SBM). Thanks to recent results on the Central Limit Theorem of the rate of convergence of estimators for the SBM, we prove a conjecture left open that the MLE has asymptotically a mixed normal distribution involving the local time with a rate of convergence of order $1/4$. We also give a series expansion of the MLE and study the asymptotic behavior of the score and its derivatives, as well as their variation with the skewness parameter. In par…
We propose a class of estimators for the parameters of a GARCH(p,q) sequence. We show that our estimators are consistent and asymptotically normal under mild conditions. The quasi-maximum likelihood and the likelihood estimators are discussed in detail. We show that the maximum likelihood estimator is optimal. If the tail of the distribution of the innovations is polynomial, even a quasi-maximum likelihood estimator based on exponential density performs better than the standard normal density-based quasi-likelihood estimator of Lee and Hansen and Lumsdaine.
In many complex statistical models maximum likelihood estimators cannot be calculated. In the paper we solve this problem using Markov chain Monte Carlo approximation of the true likelihood. In the main result we prove asymptotic normality of the estimator, when both sample sizes (the initial and Monte Carlo one) tend to infinity. Our result can be applied to models with intractable norming constants and missing data models.
This paper deals with multivariate Gaussian models for which the covariance matrix is a Kronecker product of two matrices. We consider maximum likelihood estimation of the model parameters, in particular of the covariance matrix. There is no explicit expression for the maximum likelihood estimator of a Kronecker product covariance matrix. The main question in this paper is whether the maximum likelihood estimator of the covariance matrix exists and if it is unique. The answers are different for different models that we consider.
In this note, we propose a new approach for the proof of the consistency and normality of the maximum likelihood estimator for nonlinear AR processes with markov-switching under the assumptions of uniform exponential forgetting of the prediction filter and $α$-mixing property. We show that in the linear and Gaussian case our assumptions are fully satisfied.
This paper deals with a general class of observation-driven time series models with a special focus on time series of counts. We provide conditions under which there exist strict-sense stationary and ergodic versions of such processes. The consistency of the maximum likelihood estimators is then derived for well- specified and misspecified models.
We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness s>1/2. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric c…
Some quasi-arithmetic means of random variables easily give unbiased strongly consistent closed-form estimators of the joint of the location and scale parameters of the Cauchy distribution. The one-step estimators of those quasi-arithmetic means of the Cauchy distribution are considered. We establish the Bahadur efficiency of the maximum likelihood estimator and the one-step estimators. We also show that the rate of the convergence of the mean-squared errors achieves the Cramer-Rao bound. Our results are also applicable to the circular Cauchy distribution.
This paper establishes the almost sure convergence and asymptotic normality of levels and differenced quasi maximum-likelihood (QML) estimators of dynamic panel data models. The QML estimators are robust with respect to initial conditions, conditional and time-series heteroskedasticity, and misspecification of the log-likelihood. The paper also provides an ECME algorithm for calculating levels QML estimates. Finally, it uses Monte Carlo experiments to compare the finite sample performance of levels and differenced QML estimators, the differenced GMM estimator, and the system GMM estimator. In …
The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family…
In this paper, we compare maximum likelihood (ML), quasi likelihood (QL) and weighted least squares (WLS) estimators for proportional error nonlinear regression models. Literature on thermoluminescence sedimentary dating revealed another estimator similar to weighted least squares but observed responses used as weights. This estimator that we refer to as data weighted least squares (DWLS) is also included in the comparison. We show that on the order $σ, $ all four estimators behave similar to ordinary least squares estimators for standard linear regression models. On the order of $σ^2, $ the e…
The class of observation-driven models (ODMs) includes many models of non-linear time series which, in a fashion similar to, yet different from, hidden Markov models (HMMs), involve hidden variables. Interestingly, in contrast to most HMMs, ODMs enjoy likelihoods that can be computed exactly with computational complexity of the same order as the number of observations, making maximum likelihood estimation the privileged approach for statistical inference for these models. A celebrated example of general order ODMs is the GARCH$(p,q)$ model, for which ergodicity and inference has been studied e…
This work studies the statistical properties of the maximum penalized likelihood approach in a semi-parametric framework. We recall the penalized likelihood approach for estimating a function and review some asymptotic results. We investigate the properties of two estimators of the variance of maximum penalized likelihood estimators: sandwich estimator and a Bayesian estimator. The coverage rates of confidence intervals based on these estimators are studied through a simulation study of survival data. In a first simulation the coverage rates for the survival function and the hazard function ar…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a long established result. Explicit bounds for the distributional distance between the distribution of the MLE and the normal distribution have recently been obtained for the case of independent random variables. In this paper, a local dependence structure is introduced between the random variables and we give upper bounds which are specified for the Wasserstein metric.
This paper is devoted to the estimation of a vector parametrizing an energy function associated to some "Nearest-Neighbours" Gibbs point process, via the pseudo-likelihood method. We present some convergence results concerning this estimator, that is strong consistency and asymptotic normality, when only a single realization is observed. Sufficient conditions are expressed in terms of the local energy function and are verified on some examples.
We consider the problem of estimating the joint distribution function of the event time and a continuous mark variable based on censored data. More specifically, the event time is subject to current status censoring and the continuous mark is only observed in case inspection takes place after the event time. The nonparametric maximum likelihood estimator (MLE) in this model is known to be inconsistent. We propose and study an alternative likelihood based estimator, maximizing a smoothed log-likelihood, hence called a maximum smoothed likelihood estimator (MSLE). This estimator is shown to be w…
The asymptotic normality of the maximum likelihood estimator (MLE) under regularity conditions is a cornerstone of statistical theory. In this paper, we give explicit upper bounds on the distributional distance between the distribution of the MLE of a vector parameter, and the multivariate normal distribution. We work with possibly high-dimensional, independent but not necessarily identically distributed random vectors. In addition, we obtain explicit upper bounds even in cases where the MLE cannot be expressed analytically.
A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to ze…
Max-stable distributions and processes are important models for extreme events and the assessment of tail risks. The full, multivariate likelihood of a parametric max-stable distribution is complicated and only recent advances enable its use. The asymptotic properties of the maximum likelihood estimator in multivariate extremes are mostly unknown. In this paper we provide natural conditions on the exponent function and the angular measure of the max-stable distribution that ensure asymptotic normality of the estimator. We show the effectiveness of this result by applying it to popular parametr…
Models with multiple change points are used in many fields; however, the theoretical properties of maximum likelihood estimators of such models have received relatively little attention. The goal of this paper is to establish the asymptotic properties of maximum likelihood estimators of the parameters of a multiple change-point model for a general class of models in which the form of the distribution can change from segment to segment and in which, possibly, there are parameters that are common to all segments. Consistency of the maximum likelihood estimators of the change points is establishe…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
We consider a one dimensional ballistic random walk evolving in an i.i.d. parametric random environment. We provide a maximum likelihood estimation procedure of the environment parameters based on a single observation of the path till the time it reaches a distant site, and prove that this estimator is consistent as the distant site tends to infinity. We also explore the numerical performances of our estimation procedure.
The $α$-Brownian bridge, or scaled Brownian bridge, is a generalization of the Brownian bridge with a scaling parameter that determines how strong the force that pulls the process back to 0 is. The bias of the maximum likelihood estimator of the parameter $α$ is derived and a bias-correction that improves the estimator substantially is proposed. The properties of the bias-corrected estimator and four Bayesian estimators based on non-informative priors are evaluated in a simulation study.
We consider a one dimensional ballistic random walk evolving in a parametric independent and identically distributed random environment. We study the asymptotic properties of the maximum likelihood estimator of the parameter based on a single observation of the path till the time it reaches a distant site. We prove an asymptotic normality result for this consistent estimator as the distant site tends to infinity and establish that it achieves the Cramér-Rao bound. We also explore in a simulation setting the numerical behaviour of asymptotic confidence regions for the parameter value.
The extreme value index is a fundamental parameter in univariate Extreme Value Theory (EVT). It captures the tail behavior of a distribution and is central in the extrapolation beyond observed data. Among other semi-parametric methods (such as the popular Hill's estimator), the Block Maxima (BM) and Peaks-Over-Threshold (POT) methods are widely used for assessing the extreme value index and related normalizing constants. We provide asymptotic theory for the maximum likelihood estimators (MLE) based on the BM method. Our main result is the asymptotic normality of the MLE with a non-trivial bias…
The extremes of a stationary time series typically occur in clusters. A primary measure for this phenomenon is the extremal index, representing the reciprocal of the expected cluster size. Both a disjoint and a sliding blocks estimator for the extremal index are analyzed in detail. In contrast to many competitors, the estimators only depend on the choice of one parameter sequence. We derive an asymptotic expansion, prove asymptotic normality and show consistency of an estimator for the asymptotic variance. Explicit calculations in certain models and a finite-sample Monte Carlo simulation study…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a cornerstone of statistical theory. In the present paper, we provide sharp explicit upper bounds on Zolotarev-type distances between the exact, unknown distribution of the MLE and its limiting normal distribution. Our approach to this fundamental issue is based on a sound combination of the Delta method, Stein's method, Taylor expansions and conditional expectations, for the classical situations where the MLE can be expressed as a function of a sum of independent and identically distributed terms. This encompasses in partic…
This paper considers a proportional hazards model, which allows one to examine the extent to which covariates interact nonlinearly with an exposure variable, for analysis of lifetime data. A local partial-likelihood technique is proposed to estimate nonlinear interactions. Asymptotic normality of the proposed estimator is established. The baseline hazard function, the bias and the variance of the local likelihood estimator are consistently estimated. In addition, a one-step local partial-likelihood estimator is presented to facilitate the computation of the proposed procedure and is demonstrat…
We study asymptotic properties of maximum likelihood estimators of drift parameters for a jump-type Heston model based on continuous time observations, where the jump process can be any purely non-Gaussian Lévy process of not necessarily bounded variation with a Lévy measure concentrated on $(-1,\infty)$. We prove strong consistency and asymptotic normality for all admissible parameter values except one, where we show only weak consistency and mixed normal (but non-normal) asymptotic behavior. It turns out that the volatility of the price process is a measurable function of the price process. …
We study asymptotic properties of maximum likelihood estimators for Heston models based on continuous time observations of the log-price process. We distinguish three cases: subcritical (also called ergodic), critical and supercritical. In the subcritical case, asymptotic normality is proved for all the parameters, while in the critical and supercritical cases, non-standard asymptotic behavior is described.
This paper generalizes asymptotic properties obtained in the observation-driven times series models considered by \cite{dou:kou:mou:2013} in the sense that the conditional law of each observation is also permitted to depend on the parameter. The existence of ergodic solutions and the consistency of the Maximum Likelihood Estimator (MLE) are derived under easy-to-check conditions. The obtained conditions appear to apply for a wide class of models. We illustrate our results with specific observation-driven times series, including the recently introduced NBIN-GARCH and NM-GARCH models, demonstrat…
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
Robust estimators of location and dispersion are often used in the elliptical model to obtain an uncontaminated and highly representative subsample by trimming the data outside an ellipsoid based in the associated Mahalanobis distance. Here we analyze some one (or $k$)-step Maximum Likelihood Estimators computed on a subsample obtained with such a procedure. We introduce different models which arise naturally from the ways in which the discarded data can be treated, leading to truncated or censored likelihoods, as well as to a likelihood based on an only outliers gross errors model. Results on…
For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to $α$-divergence, which includes the special cases of Kullback--Leibler divergence, the Hellinger distance and $χ^2$ divergence. The asymptotic expansion of the risk is given with respect to sample sizes of up to order $n^{-2}$. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model. We attempt to measure the difficulty of specifying a model through this expansion.
The three-parameter generalized extreme value distribution arises from classical univariate extreme value theory and is in common use for analyzing the far tail of observed phenomena. Curiously, important asymptotic properties of likelihood-based estimation under this standard model have yet to be established. In this paper, we formally prove that the maximum likelihood estimator is global and unique. An interesting secondary result entails the uniform consistency of a class of limit relations in a tight neighborhood of the shape parameter.
Consider an $(L,1)$ random walk in an i.i.d. random environment, whose environment involves certain parameter. We get the maximum likelihood estimator(MLE) of the environment parameter which can be written as functionals of a multitype branching process with immigration in a random environment(BPIRE). Because the offspring distributions of the involved multitype BPIRE are of the linear fractional type, the limit invariant distribution of the multitype BPIRE can be computed explicitly. As a result, we get the consistency of the MLE. Our result is a generalization of Comets et al. [Stochastic Pr…
