Randal Douc, Paul Doukhan, Eric Moulines
This paper deals with a general class of observation-driven time series models with a special focus on time series of counts. We provide conditions under which there exist strict-sense stationary and ergodic versions of such processes. The consistency of the maximum likelihood estimators is then derived for well- specified and misspecified models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
The class of observation-driven models (ODMs) includes many models of non-linear time series which, in a fashion similar to, yet different from, hidden Markov models (HMMs), involve hidden variables. Interestingly, in contrast to most HMMs, ODMs enjoy likelihoods that can be computed exactly with computational complexity of the same order as the number of observations, making maximum likelihood estimation the privileged approach for statistical inference for these models. A celebrated example of general order ODMs is the GARCH$(p,q)$ model, for which ergodicity and inference has been studied e…
We prove uniqueness of the maximum likelihood estimator for the class of k-monotone densities.
Composite likelihoods are a class of alternatives to the full likelihood which are widely used in many situations in which the likelihood itself is intractable. A composite likelihood may be computed without the need to specify the full distribution of the response, which means that in some situations the resulting estimator will be more robust to model misspecification than the maximum likelihood estimator. The purpose of this note is to show that such increased robustness is not guaranteed. An example is given in which various marginal composite likelihood estimators are inconsistent under m…
We address estimation of parametric coefficients of a pure-jump Lévy driven univariate stochastic differential equation (SDE) model, which is observed at high frequency over a fixed time period. It is known from the previous study Masuda (2013) that adopting the conventional Gaussian quasi-maximum likelihood estimator then leads to an inconsistent estimator. In this paper, under the assumption that the driving Lévy process is locally stable, we extend the Gaussian framework into a non-Gaussian counterpart, by introducing a novel quasi-likelihood function formally based on the small-time stable…
The use of {\it Mathematica} in deriving mean likelihood estimators is discussed. Comparisons between the maximum likelihood estimator, the mean likelihood estimator and the Bayes estimate based on a Jeffrey's noninformative prior using the criteria mean-square error and Pitman measure of closeness. Based on these criteria we find that for the first-order moving-average time series model, the mean likelihood estimator outperforms the maximum likelihood estimator and the Bayes estimator with a Jeffrey's noninformative prior. {\it Mathematica} was used for symbolic and numeric computations as we…
Upper and lower class functions for the maximum likelihood estimator of the arrival and the service rates in a $GI/G/1$ queue are studied and the results are verified for M/M/1 queue.
Probabilistic cellular automata with local transition matrices: synchronization, ergodicity, and inference
本文提出一类可展现同步与遍历性的概率元胞自动机，给出局部转移矩阵的充要条件，并设计最小二乘估计器从多种数据推断该矩阵，证明渐近正态性并提供非渐近精度界。
Maximum likelihood estimation has been extensively used in the joint analysis of repeated measurements and survival time. However, there is a lack of theoretical justification of the asymptotic properties for the maximum likelihood estimators. This paper intends to fill this gap. Specifically, we prove the consistency of the maximum likelihood estimators and derive their asymptotic distributions. The maximum likelihood estimators are shown to be semiparametrically efficient.
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
We propose a class of estimators for the parameters of a GARCH(p,q) sequence. We show that our estimators are consistent and asymptotically normal under mild conditions. The quasi-maximum likelihood and the likelihood estimators are discussed in detail. We show that the maximum likelihood estimator is optimal. If the tail of the distribution of the innovations is polynomial, even a quasi-maximum likelihood estimator based on exponential density performs better than the standard normal density-based quasi-likelihood estimator of Lee and Hansen and Lumsdaine.
In many complex statistical models maximum likelihood estimators cannot be calculated. In the paper we solve this problem using Markov chain Monte Carlo approximation of the true likelihood. In the main result we prove asymptotic normality of the estimator, when both sample sizes (the initial and Monte Carlo one) tend to infinity. Our result can be applied to models with intractable norming constants and missing data models.
Asymptotics of the maximum likelihood estimator of the location parameter of Pearson Type VII distribution
研究Pearson VII分布已知尺度下位置参数最大似然估计的渐进行为，包括强一致性、渐近正态性、Bahadur效率和渐近方差。重点分析重尾情况，如Cauchy分布，克服似然方程多根难题。
This paper deals with multivariate Gaussian models for which the covariance matrix is a Kronecker product of two matrices. We consider maximum likelihood estimation of the model parameters, in particular of the covariance matrix. There is no explicit expression for the maximum likelihood estimator of a Kronecker product covariance matrix. The main question in this paper is whether the maximum likelihood estimator of the covariance matrix exists and if it is unique. The answers are different for different models that we consider.
In this note, we propose a new approach for the proof of the consistency and normality of the maximum likelihood estimator for nonlinear AR processes with markov-switching under the assumptions of uniform exponential forgetting of the prediction filter and $α$-mixing property. We show that in the linear and Gaussian case our assumptions are fully satisfied.
We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness s>1/2. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric c…
Some quasi-arithmetic means of random variables easily give unbiased strongly consistent closed-form estimators of the joint of the location and scale parameters of the Cauchy distribution. The one-step estimators of those quasi-arithmetic means of the Cauchy distribution are considered. We establish the Bahadur efficiency of the maximum likelihood estimator and the one-step estimators. We also show that the rate of the convergence of the mean-squared errors achieves the Cramer-Rao bound. Our results are also applicable to the circular Cauchy distribution.
This paper establishes the almost sure convergence and asymptotic normality of levels and differenced quasi maximum-likelihood (QML) estimators of dynamic panel data models. The QML estimators are robust with respect to initial conditions, conditional and time-series heteroskedasticity, and misspecification of the log-likelihood. The paper also provides an ECME algorithm for calculating levels QML estimates. Finally, it uses Monte Carlo experiments to compare the finite sample performance of levels and differenced QML estimators, the differenced GMM estimator, and the system GMM estimator. In …
The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family…
In this paper, we compare maximum likelihood (ML), quasi likelihood (QL) and weighted least squares (WLS) estimators for proportional error nonlinear regression models. Literature on thermoluminescence sedimentary dating revealed another estimator similar to weighted least squares but observed responses used as weights. This estimator that we refer to as data weighted least squares (DWLS) is also included in the comparison. We show that on the order $σ, $ all four estimators behave similar to ordinary least squares estimators for standard linear regression models. On the order of $σ^2, $ the e…
This work studies the statistical properties of the maximum penalized likelihood approach in a semi-parametric framework. We recall the penalized likelihood approach for estimating a function and review some asymptotic results. We investigate the properties of two estimators of the variance of maximum penalized likelihood estimators: sandwich estimator and a Bayesian estimator. The coverage rates of confidence intervals based on these estimators are studied through a simulation study of survival data. In a first simulation the coverage rates for the survival function and the hazard function ar…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a long established result. Explicit bounds for the distributional distance between the distribution of the MLE and the normal distribution have recently been obtained for the case of independent random variables. In this paper, a local dependence structure is introduced between the random variables and we give upper bounds which are specified for the Wasserstein metric.
A Proof of Strong Consistency of Maximum Likelihood Estimator for Independent Non-Identically Distributed Data
该研究在密度函数满足特定假设的条件下，基于已有研究给出独立非同分布数据下极大似然估计量强相合性的通用证明，并将其用于证明轨道确定问题中极大似然估计量的强相合性。
This paper is devoted to the estimation of a vector parametrizing an energy function associated to some "Nearest-Neighbours" Gibbs point process, via the pseudo-likelihood method. We present some convergence results concerning this estimator, that is strong consistency and asymptotic normality, when only a single realization is observed. Sufficient conditions are expressed in terms of the local energy function and are verified on some examples.
Asymptotic properties of maximum likelihood estimators for determinantal point processes
该研究推导出行列式点过程（DPPs）最大似然估计量的强相合性与Berry-Esseen型界，开发了相应数值估计算法并完成模拟，还给出了2×2分块行列式矩阵最大似然估计的显式公式与讨论，完善了相关前期研究。
Ergodicity and High-Frequency Inference for Hybrid Switching Lévy-Driven Stochastic Differential Equations
针对带纯跳噪声和状态依赖切换率的混合切换Lévy驱动随机微分方程，提出三阶段推断过程，证明加权指数遍历性，并建立估计量的一致性、联合渐近正态性和大偏差不等式。
The asymptotic normality of the maximum likelihood estimator (MLE) under regularity conditions is a cornerstone of statistical theory. In this paper, we give explicit upper bounds on the distributional distance between the distribution of the MLE of a vector parameter, and the multivariate normal distribution. We work with possibly high-dimensional, independent but not necessarily identically distributed random vectors. In addition, we obtain explicit upper bounds even in cases where the MLE cannot be expressed analytically.
A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to ze…
Max-stable distributions and processes are important models for extreme events and the assessment of tail risks. The full, multivariate likelihood of a parametric max-stable distribution is complicated and only recent advances enable its use. The asymptotic properties of the maximum likelihood estimator in multivariate extremes are mostly unknown. In this paper we provide natural conditions on the exponent function and the angular measure of the max-stable distribution that ensure asymptotic normality of the estimator. We show the effectiveness of this result by applying it to popular parametr…
Models with multiple change points are used in many fields; however, the theoretical properties of maximum likelihood estimators of such models have received relatively little attention. The goal of this paper is to establish the asymptotic properties of maximum likelihood estimators of the parameters of a multiple change-point model for a general class of models in which the form of the distribution can change from segment to segment and in which, possibly, there are parameters that are common to all segments. Consistency of the maximum likelihood estimators of the change points is establishe…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
We consider a one dimensional ballistic random walk evolving in an i.i.d. parametric random environment. We provide a maximum likelihood estimation procedure of the environment parameters based on a single observation of the path till the time it reaches a distant site, and prove that this estimator is consistent as the distant site tends to infinity. We also explore the numerical performances of our estimation procedure.
Pseudo-likelihood Estimators for Graphical Models: Existence and Uniqueness
本文研究了无向高斯图模型中两类伪似然估计量存在性与唯一性的长期开放问题，给出了最小样本量的精确答案，并将其与高斯最大似然估计的解相联系。
We consider a one dimensional ballistic random walk evolving in a parametric independent and identically distributed random environment. We study the asymptotic properties of the maximum likelihood estimator of the parameter based on a single observation of the path till the time it reaches a distant site. We prove an asymptotic normality result for this consistent estimator as the distant site tends to infinity and establish that it achieves the Cramér-Rao bound. We also explore in a simulation setting the numerical behaviour of asymptotic confidence regions for the parameter value.
The extreme value index is a fundamental parameter in univariate Extreme Value Theory (EVT). It captures the tail behavior of a distribution and is central in the extrapolation beyond observed data. Among other semi-parametric methods (such as the popular Hill's estimator), the Block Maxima (BM) and Peaks-Over-Threshold (POT) methods are widely used for assessing the extreme value index and related normalizing constants. We provide asymptotic theory for the maximum likelihood estimators (MLE) based on the BM method. Our main result is the asymptotic normality of the MLE with a non-trivial bias…
The extremes of a stationary time series typically occur in clusters. A primary measure for this phenomenon is the extremal index, representing the reciprocal of the expected cluster size. Both a disjoint and a sliding blocks estimator for the extremal index are analyzed in detail. In contrast to many competitors, the estimators only depend on the choice of one parameter sequence. We derive an asymptotic expansion, prove asymptotic normality and show consistency of an estimator for the asymptotic variance. Explicit calculations in certain models and a finite-sample Monte Carlo simulation study…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a cornerstone of statistical theory. In the present paper, we provide sharp explicit upper bounds on Zolotarev-type distances between the exact, unknown distribution of the MLE and its limiting normal distribution. Our approach to this fundamental issue is based on a sound combination of the Delta method, Stein's method, Taylor expansions and conditional expectations, for the classical situations where the MLE can be expressed as a function of a sum of independent and identically distributed terms. This encompasses in partic…
This paper considers a proportional hazards model, which allows one to examine the extent to which covariates interact nonlinearly with an exposure variable, for analysis of lifetime data. A local partial-likelihood technique is proposed to estimate nonlinear interactions. Asymptotic normality of the proposed estimator is established. The baseline hazard function, the bias and the variance of the local likelihood estimator are consistently estimated. In addition, a one-step local partial-likelihood estimator is presented to facilitate the computation of the proposed procedure and is demonstrat…
This paper generalizes asymptotic properties obtained in the observation-driven times series models considered by \cite{dou:kou:mou:2013} in the sense that the conditional law of each observation is also permitted to depend on the parameter. The existence of ergodic solutions and the consistency of the Maximum Likelihood Estimator (MLE) are derived under easy-to-check conditions. The obtained conditions appear to apply for a wide class of models. We illustrate our results with specific observation-driven times series, including the recently introduced NBIN-GARCH and NM-GARCH models, demonstrat…
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
Robust estimators of location and dispersion are often used in the elliptical model to obtain an uncontaminated and highly representative subsample by trimming the data outside an ellipsoid based in the associated Mahalanobis distance. Here we analyze some one (or $k$)-step Maximum Likelihood Estimators computed on a subsample obtained with such a procedure. We introduce different models which arise naturally from the ways in which the discarded data can be treated, leading to truncated or censored likelihoods, as well as to a likelihood based on an only outliers gross errors model. Results on…
For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to $α$-divergence, which includes the special cases of Kullback--Leibler divergence, the Hellinger distance and $χ^2$ divergence. The asymptotic expansion of the risk is given with respect to sample sizes of up to order $n^{-2}$. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model. We attempt to measure the difficulty of specifying a model through this expansion.
We consider the Fisher-Snedecor diffusion; that is, the Kolmogorov-Pearson diffusion with the Fisher-Snedecor invariant distribution. In the nonstationary setting, we give explicit quantitative rates for the convergence rate of respective finite-dimensional distributions to that of the stationary Fisher-Snedecor diffusion, and for the $β$-mixing coefficient of this diffusion. As an application, we prove the law of large numbers and the central limit theorem for additive functionals of the Fisher-Snedecor diffusion and construct $P$-consistent and asymptotically normal estimators for the parame…
Consider an $(L,1)$ random walk in an i.i.d. random environment, whose environment involves certain parameter. We get the maximum likelihood estimator(MLE) of the environment parameter which can be written as functionals of a multitype branching process with immigration in a random environment(BPIRE). Because the offspring distributions of the involved multitype BPIRE are of the linear fractional type, the limit invariant distribution of the multitype BPIRE can be computed explicitly. As a result, we get the consistency of the MLE. Our result is a generalization of Comets et al. [Stochastic Pr…
This paper investigates the asymptotic theory of the quasi-maximum exponential likelihood estimators (QMELE) for ARMA--GARCH models. Under only a fractional moment condition, the strong consistency and the asymptotic normality of the global self-weighted QMELE are obtained. Based on this self-weighted QMELE, the local QMELE is showed to be asymptotically normal for the ARMA model with GARCH (finite variance) and IGARCH errors. A formal comparison of two estimators is given for some cases. A simulation study is carried out to assess the performance of these estimators, and a real example on the…
Quasi-likelihood analysis for adaptive estimation of a degenerate diffusion process
本文针对退化扩散过程提出自适应拟似然分析方法，并证明拟最大似然估计与拟贝叶斯估计在自适应方案下具有渐近正态性和矩收敛性。
The Hawks process is a point process with a self-exciting property. It has been used to model earthquakes, social media events, infections, etc., and is getting a lot of attention. However, as a real problem, there are often situations where we can not obtain data with sufficient observation time. In such cases, it is not appropriate to approximate the error distribution of an estimator by the normal distribution. To overcome this problem, we provide a rigorous mathematical foundation of the theory for the higher-order asymptotic behavior of the one-dimensional Hawkes process with an exponenti…
We present theoretical properties of the log-concave maximum likelihood estimator of a density based on an independent and identically distributed sample in $\mathbb{R}^d$. Our study covers both the case where the true underlying density is log-concave, and where this model is misspecified. We begin by showing that for a sequence of log-concave densities, convergence in distribution implies much stronger types of convergence -- in particular, it implies convergence in Hellinger distance and even in certain exponentially weighted total variation norms. In our main result, we prove the existence…
Rational Maximum Likelihood Estimators of Kronecker Covariance Matrices
本文针对具有 Kronecker 协方差结构的多元正态模型，发现在样本量与因子维度互质的特殊设定下，似然方程代数次数为 1，从而得出极大似然估计量的显式有理公式，并证明了该特殊设定外的部分逆命题。
The normality assumption on data set is very restrictive approach for modelling. The generalized form of normal distribution, named as an exponential power (EP) distribution, and its scale mixture form have been considered extensively to overcome the problem for modelling non-normal data set since last decades. However, examining the robustness properties of maximum likelihood (ML) estimators of parameters in these distributions, such as the in uence function, gross-error sensitivity, breakdown point and information-standardized sensitivity, has not been considered together. The well-known asy…
We use the delta method and Stein's method to derive, under regularity conditions, explicit upper bounds for the distributional distance between the distribution of the maximum likelihood estimator (MLE) of a $d$-dimensional parameter and its asymptotic multivariate normal distribution. Our bounds apply in situations in which the MLE can be written as a function of a sum of i.i.d. $t$-dimensional random vectors. We apply our general bound to establish a bound for the multivariate normal approximation of the MLE of the normal distribution with unknown mean and variance.
We have shown in previous work that statistical inference for cooperative sequential adsorption model can be based on maximum likelihood estimation. In this paper we continue this research and establish asymptotic normality of the maximum likelihood estimator in thermodynamic limit. We also perform and discuss some numerical simulations of the model.
A trigonometrically approximated maximum likelihood estimation for $α$-stable laws is proposed. The estimator solves the approximated likelihood equation, which is obtained by projecting a true score function on the space spanned by trigonometric functions. The projected score is expressed only by real and imaginary parts of the characteristic function and their derivatives, so that we can explicitly construct the targeting estimating equation. We study the asymptotic properties of the proposed estimator and show consistency and asymptotic normality. Furthermore, as the number of trigonometric…
This paper gives a new approach for the maximum likelihood estimation of the joint of the location and scale of the Cauchy distribution. We regard the joint as a single complex parameter and derive a new form of the likelihood equation of a complex variable. Based on the equation, we provide a new iterative scheme approximating the maximum likelihood estimate. We also handle the equation in an algebraic manner and derive a polynomial containing the maximum likelihood estimate as a root. This algebraic approach provides another scheme approximating the maximum likelihood estimate by root-findin…
We consider the problem of estimating the joint distribution function of the event time and a continuous mark variable based on censored data. More specifically, the event time is subject to current status censoring and the continuous mark is only observed in case inspection takes place after the event time. The nonparametric maximum likelihood estimator (MLE) in this model is known to be inconsistent. We propose and study an alternative likelihood based estimator, maximizing a smoothed log-likelihood, hence called a maximum smoothed likelihood estimator (MSLE). This estimator is shown to be w…
In finite mixtures of location-scale distributions, if there is no constraint or penalty on the parameters, then the maximum likelihood estimator does not exist because the likelihood is unbounded. To avoid this problem, we consider a penalized likelihood, where the penalty is a function of the minimum of the ratios of the scale parameters and the sample size. It is shown that the penalized maximum likelihood estimator is strongly consistent. We also analyze the consistency of a penalized maximum likelihood estimator where the penalty is imposed on the scale parameters themselves.
We consider the problem of estimating the distribution function, the density and the hazard rate of the (unobservable) event time in the current status model. A well studied and natural nonparametric estimator for the distribution function in this model is the nonparametric maximum likelihood estimator (MLE). We study two alternative methods for the estimation of the distribution function, assuming some smoothness of the event time distribution. The first estimator is based on a maximum smoothed likelihood approach. The second method is based on smoothing the (discrete) MLE of the distribution…
This paper presents asymptotic properties of the maximum pseudo-likelihood estimator of a vector $\Vectθ$ parameterizing a stationary Gibbs point process. Sufficient conditions, expressed in terms of the local energy function defining a Gibbs point process, to establish strong consistency and asymptotic normality results of this estimator depending on a single realization, are presented.These results are general enough to no longer require the local stability and the linearity in terms of the parameters of the local energy function. We consider characteristic examples of such models, the Lenna…
Maximum Likelihood Estimation of the Parameters of Matrix Variate Symmetric Laplace Distribution
本文研究多元对称拉普拉斯分布向矩阵变量情形的扩展，提出基于EM算法求解无法显式获得的参数最大似然估计量，给出MLE存在条件，并通过模拟数据评估了估计量的偏差与离散度。
We consider nonsynchronous sampling of parameterized stochastic regression models, which contain stochastic differential equations. Constructing a quasi-likelihood function, we prove that the quasi-maximum likelihood estimator and the Bayes type estimator are consistent and asymptotically mixed normal when the sampling frequency of the nonsynchronous data becomes large.
This paper deals with the problem of global parameter estimation of affine diffusions in $\mathbb{R}_+ \times \mathbb{R}^n$ denoted by $AD(1, n)$ where $n$ is a positive integer which is a subclass of affine diffusions introduced by Duffie et al in [14]. The $AD(1, n)$ model can be applied to the pricing of bond and stock options, which is illustrated for the Vasicek, Cox-Ingersoll-Ross and Heston models. Our first result is about the classification of $AD(1, n)$ processes according to the subcritical, critical and supercritical cases. Then, we give the stationarity and the ergodicity theorems…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。