In making inference on the relation between failure and exposure histories in the Cox semiparametric model, the maximum partial likelihood estimator (MPLE) of the finite dimensional odds parameter, and the Breslow estimator of the baseline survival function, are known to achieve full efficiency when data is available for all time on all cohort members, even when the covariates are time dependent. When cohort sizes become too large for the collection of complete data, sampling schemes such as nested case control sampling must be used and, under various models, there exist estimators based on the same information as the MPLE having smaller asymptotic variance. Though the MPLE is therefore not efficient under sampling in general, it approaches efficiency in highly stratified situations, or instances where the covariate values are increasingly less dependent upon the past, when the covariate distribution, not depending on the real parameter of interest, is unknown and there is no censoring. In particular, in such situations, when using the nested case control sampling design, both the MPLE and the Breslow estimator of the baseline survival function achieve the information lower bound both in the distributional and the minimax senses in the limit as the number of cohort members tends to infinity.
The main idea of nested sampling is to substitute the high-dimensional likelihood integral over the parameter space $Ω$ by an integral over the unit line $[0,1]$ by employing a push-forward with respect to a suitable transformation. For this substitution, it is often implicitly or explicitly assumed that samples from the prior are uniformly distributed along this unit line after having been mapped by this transformation. We show that this assumption is wrong, especially in the case of a likelihood function with plateaus. Nevertheless, we show that the substitution enacted by nested sampling wo…
In statistics, there are a variety of methods for performing model selection that all stem from slightly different paradigms of statistical inference. The reasons for choosing one particular method over another seem to be based entirely on philosophical preferences. In the case of non-nested model selection, two of the prevailing techniques are the Bayes Factor and the likelihood ratio. This article focuses on reconciling the likelihood ratio and the Bayes Factor for comparing a pair of non-nested models under two different problem frameworks typical in forensic science, the common-source and …
Regularizing nested Monte Carlo Sobol' index estimators to balance the trade-off between explorations and repetitions in global sensitivity analysis of stochastic models
Coarse structural nested mean models are used to estimate treatment effects from longitudinal observational data. Coarse structural nested mean models lead to a large class of estimators. It turns out that estimates and standard errors may differ considerably within this class. We prove that, under additional assumptions, there exists an explicit solution for the optimal estimator within the class of coarse structural nested mean models. Moreover, we show that even if the additional assumptions do not hold, this optimal estimator is doubly-robust: it is consistent and asymptotically normal not…
Composite likelihoods are a class of alternatives to the full likelihood which are widely used in many situations in which the likelihood itself is intractable. A composite likelihood may be computed without the need to specify the full distribution of the response, which means that in some situations the resulting estimator will be more robust to model misspecification than the maximum likelihood estimator. The purpose of this note is to show that such increased robustness is not guaranteed. An example is given in which various marginal composite likelihood estimators are inconsistent under m…
The use of {\it Mathematica} in deriving mean likelihood estimators is discussed. Comparisons between the maximum likelihood estimator, the mean likelihood estimator and the Bayes estimate based on a Jeffrey's noninformative prior using the criteria mean-square error and Pitman measure of closeness. Based on these criteria we find that for the first-order moving-average time series model, the mean likelihood estimator outperforms the maximum likelihood estimator and the Bayes estimator with a Jeffrey's noninformative prior. {\it Mathematica} was used for symbolic and numeric computations as we…
Nested sampling is a simulation method for approximating marginal likelihoods proposed by Skilling (2006). We establish that nested sampling has an approximation error that vanishes at the standard Monte Carlo rate and that this error is asymptotically Gaussian. We show that the asymptotic variance of the nested sampling approximation typically grows linearly with the dimension of the parameter. We discuss the applicability and efficiency of nested sampling in realistic problems, and we compare it with two current methods for computing marginal likelihood. We propose an extension that avoids r…
Upper and lower class functions for the maximum likelihood estimator of the arrival and the service rates in a $GI/G/1$ queue are studied and the results are verified for M/M/1 queue.
Maximum likelihood estimation has been extensively used in the joint analysis of repeated measurements and survival time. However, there is a lack of theoretical justification of the asymptotic properties for the maximum likelihood estimators. This paper intends to fill this gap. Specifically, we prove the consistency of the maximum likelihood estimators and derive their asymptotic distributions. The maximum likelihood estimators are shown to be semiparametrically efficient.
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
We propose a class of estimators for the parameters of a GARCH(p,q) sequence. We show that our estimators are consistent and asymptotically normal under mild conditions. The quasi-maximum likelihood and the likelihood estimators are discussed in detail. We show that the maximum likelihood estimator is optimal. If the tail of the distribution of the innovations is polynomial, even a quasi-maximum likelihood estimator based on exponential density performs better than the standard normal density-based quasi-likelihood estimator of Lee and Hansen and Lumsdaine.
In many complex statistical models maximum likelihood estimators cannot be calculated. In the paper we solve this problem using Markov chain Monte Carlo approximation of the true likelihood. In the main result we prove asymptotic normality of the estimator, when both sample sizes (the initial and Monte Carlo one) tend to infinity. Our result can be applied to models with intractable norming constants and missing data models.
This paper deals with multivariate Gaussian models for which the covariance matrix is a Kronecker product of two matrices. We consider maximum likelihood estimation of the model parameters, in particular of the covariance matrix. There is no explicit expression for the maximum likelihood estimator of a Kronecker product covariance matrix. The main question in this paper is whether the maximum likelihood estimator of the covariance matrix exists and if it is unique. The answers are different for different models that we consider.
In this note, we propose a new approach for the proof of the consistency and normality of the maximum likelihood estimator for nonlinear AR processes with markov-switching under the assumptions of uniform exponential forgetting of the prediction filter and $α$-mixing property. We show that in the linear and Gaussian case our assumptions are fully satisfied.
This paper deals with a general class of observation-driven time series models with a special focus on time series of counts. We provide conditions under which there exist strict-sense stationary and ergodic versions of such processes. The consistency of the maximum likelihood estimators is then derived for well- specified and misspecified models.
We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness s>1/2. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric c…
The stochastic block model (SBM) is a probabilistic model de- signed to describe heterogeneous directed and undirected graphs. In this paper, we address the asymptotic inference on SBM by use of maximum- likelihood and variational approaches. The identi ability of SBM is proved, while asymptotic properties of maximum-likelihood and variational esti- mators are provided. In particular, the consistency of these estimators is settled, which is, to the best of our knowledge, the rst result of this type for variational estimators with random graphs.
Some quasi-arithmetic means of random variables easily give unbiased strongly consistent closed-form estimators of the joint of the location and scale parameters of the Cauchy distribution. The one-step estimators of those quasi-arithmetic means of the Cauchy distribution are considered. We establish the Bahadur efficiency of the maximum likelihood estimator and the one-step estimators. We also show that the rate of the convergence of the mean-squared errors achieves the Cramer-Rao bound. Our results are also applicable to the circular Cauchy distribution.
This paper establishes the almost sure convergence and asymptotic normality of levels and differenced quasi maximum-likelihood (QML) estimators of dynamic panel data models. The QML estimators are robust with respect to initial conditions, conditional and time-series heteroskedasticity, and misspecification of the log-likelihood. The paper also provides an ECME algorithm for calculating levels QML estimates. Finally, it uses Monte Carlo experiments to compare the finite sample performance of levels and differenced QML estimators, the differenced GMM estimator, and the system GMM estimator. In …
We study the problem of estimating the covariance parameters of a one-dimensional Gaussian process with exponential covariance function under fixed-domain asymptotics. We show that the weighted pairwise maximum likelihood estimator of the microergodic parameter can be consistent or inconsistent. This depends on the range of admissible parameter values in the likelihood optimization. On the other hand, the weighted pairwise conditional maximum likelihood estimator is always consistent. Both estimators are also asymptotically Gaussian when they are consistent. Their asymptotic variances are larg…
This paper establishes asymptotic results for the maximum likelihood and restricted maximum likelihood (REML) estimators of the parameters in the nested error regression model for clustered data when both of the number of independent clusters and the cluster sizes (the number of observations in each cluster) go to infinity. Under very mild conditions, the estimators are shown to be asymptotically normal with an elegantly structured covariance matrix. There are no restrictions on the rate at which the cluster size tends to infinity but it turns out that we need to treat within cluster parameter…
The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family…
In this paper, we compare maximum likelihood (ML), quasi likelihood (QL) and weighted least squares (WLS) estimators for proportional error nonlinear regression models. Literature on thermoluminescence sedimentary dating revealed another estimator similar to weighted least squares but observed responses used as weights. This estimator that we refer to as data weighted least squares (DWLS) is also included in the comparison. We show that on the order $σ, $ all four estimators behave similar to ordinary least squares estimators for standard linear regression models. On the order of $σ^2, $ the e…
This work studies the statistical properties of the maximum penalized likelihood approach in a semi-parametric framework. We recall the penalized likelihood approach for estimating a function and review some asymptotic results. We investigate the properties of two estimators of the variance of maximum penalized likelihood estimators: sandwich estimator and a Bayesian estimator. The coverage rates of confidence intervals based on these estimators are studied through a simulation study of survival data. In a first simulation the coverage rates for the survival function and the hazard function ar…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a long established result. Explicit bounds for the distributional distance between the distribution of the MLE and the normal distribution have recently been obtained for the case of independent random variables. In this paper, a local dependence structure is introduced between the random variables and we give upper bounds which are specified for the Wasserstein metric.
This paper is devoted to the estimation of a vector parametrizing an energy function associated to some "Nearest-Neighbours" Gibbs point process, via the pseudo-likelihood method. We present some convergence results concerning this estimator, that is strong consistency and asymptotic normality, when only a single realization is observed. Sufficient conditions are expressed in terms of the local energy function and are verified on some examples.
Robust estimation under multivariate normal (MVN) mixture model is always a computational challenge. A recently proposed maximum pseudo \b{eta}-likelihood estimator aims to estimate the unknown parameters of a MVN mixture model in the spirit of minimum density power divergence (DPD) methodology but with a relatively simpler and tractable computational algorithm even for larger dimensions. In this letter, we will rigorously derive the existence and weak consistency of the maximum pseudo \b{eta}-likelihood estimator in case of MVN mixture models under a reasonable set of assumptions.
The asymptotic normality of the maximum likelihood estimator (MLE) under regularity conditions is a cornerstone of statistical theory. In this paper, we give explicit upper bounds on the distributional distance between the distribution of the MLE of a vector parameter, and the multivariate normal distribution. We work with possibly high-dimensional, independent but not necessarily identically distributed random vectors. In addition, we obtain explicit upper bounds even in cases where the MLE cannot be expressed analytically.
A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to ze…
Max-stable distributions and processes are important models for extreme events and the assessment of tail risks. The full, multivariate likelihood of a parametric max-stable distribution is complicated and only recent advances enable its use. The asymptotic properties of the maximum likelihood estimator in multivariate extremes are mostly unknown. In this paper we provide natural conditions on the exponent function and the angular measure of the max-stable distribution that ensure asymptotic normality of the estimator. We show the effectiveness of this result by applying it to popular parametr…
Models with multiple change points are used in many fields; however, the theoretical properties of maximum likelihood estimators of such models have received relatively little attention. The goal of this paper is to establish the asymptotic properties of maximum likelihood estimators of the parameters of a multiple change-point model for a general class of models in which the form of the distribution can change from segment to segment and in which, possibly, there are parameters that are common to all segments. Consistency of the maximum likelihood estimators of the change points is establishe…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
We consider the asymptotic behaviour of the marginal maximum likelihood empirical Bayes posterior distribution in general setting. First we characterize the set where the maximum marginal likelihood estimator is located with high probability. Then we provide oracle type of upper and lower bounds for the contraction rates of the empirical Bayes posterior. We also show that the hierarchical Bayes posterior achieves the same contraction rate as the maximum marginal likelihood empirical Bayes posterior. We demonstrate the applicability of our general results for various models and prior distributi…
We consider a one dimensional ballistic random walk evolving in an i.i.d. parametric random environment. We provide a maximum likelihood estimation procedure of the environment parameters based on a single observation of the path till the time it reaches a distant site, and prove that this estimator is consistent as the distant site tends to infinity. We also explore the numerical performances of our estimation procedure.
We consider a one dimensional ballistic random walk evolving in a parametric independent and identically distributed random environment. We study the asymptotic properties of the maximum likelihood estimator of the parameter based on a single observation of the path till the time it reaches a distant site. We prove an asymptotic normality result for this consistent estimator as the distant site tends to infinity and establish that it achieves the Cramér-Rao bound. We also explore in a simulation setting the numerical behaviour of asymptotic confidence regions for the parameter value.
The extreme value index is a fundamental parameter in univariate Extreme Value Theory (EVT). It captures the tail behavior of a distribution and is central in the extrapolation beyond observed data. Among other semi-parametric methods (such as the popular Hill's estimator), the Block Maxima (BM) and Peaks-Over-Threshold (POT) methods are widely used for assessing the extreme value index and related normalizing constants. We provide asymptotic theory for the maximum likelihood estimators (MLE) based on the BM method. Our main result is the asymptotic normality of the MLE with a non-trivial bias…
The extremes of a stationary time series typically occur in clusters. A primary measure for this phenomenon is the extremal index, representing the reciprocal of the expected cluster size. Both a disjoint and a sliding blocks estimator for the extremal index are analyzed in detail. In contrast to many competitors, the estimators only depend on the choice of one parameter sequence. We derive an asymptotic expansion, prove asymptotic normality and show consistency of an estimator for the asymptotic variance. Explicit calculations in certain models and a finite-sample Monte Carlo simulation study…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a cornerstone of statistical theory. In the present paper, we provide sharp explicit upper bounds on Zolotarev-type distances between the exact, unknown distribution of the MLE and its limiting normal distribution. Our approach to this fundamental issue is based on a sound combination of the Delta method, Stein's method, Taylor expansions and conditional expectations, for the classical situations where the MLE can be expressed as a function of a sum of independent and identically distributed terms. This encompasses in partic…
This paper considers a proportional hazards model, which allows one to examine the extent to which covariates interact nonlinearly with an exposure variable, for analysis of lifetime data. A local partial-likelihood technique is proposed to estimate nonlinear interactions. Asymptotic normality of the proposed estimator is established. The baseline hazard function, the bias and the variance of the local likelihood estimator are consistently estimated. In addition, a one-step local partial-likelihood estimator is presented to facilitate the computation of the proposed procedure and is demonstrat…
This paper generalizes asymptotic properties obtained in the observation-driven times series models considered by \cite{dou:kou:mou:2013} in the sense that the conditional law of each observation is also permitted to depend on the parameter. The existence of ergodic solutions and the consistency of the Maximum Likelihood Estimator (MLE) are derived under easy-to-check conditions. The obtained conditions appear to apply for a wide class of models. We illustrate our results with specific observation-driven times series, including the recently introduced NBIN-GARCH and NM-GARCH models, demonstrat…
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
Robust estimators of location and dispersion are often used in the elliptical model to obtain an uncontaminated and highly representative subsample by trimming the data outside an ellipsoid based in the associated Mahalanobis distance. Here we analyze some one (or $k$)-step Maximum Likelihood Estimators computed on a subsample obtained with such a procedure. We introduce different models which arise naturally from the ways in which the discarded data can be treated, leading to truncated or censored likelihoods, as well as to a likelihood based on an only outliers gross errors model. Results on…
For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to $α$-divergence, which includes the special cases of Kullback--Leibler divergence, the Hellinger distance and $χ^2$ divergence. The asymptotic expansion of the risk is given with respect to sample sizes of up to order $n^{-2}$. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model. We attempt to measure the difficulty of specifying a model through this expansion.
The maximum composite likelihood estimator for parametric models of determinantal point processes (DPPs) is discussed. Since the joint intensities of these point processes are given by determinant of positive definite kernels, we have the explicit form of the joint intensities for every order. This fact enables us to consider the generalized maximum composite likelihood estimator for any order. This paper introduces the two step generalized composite likelihood estimator and shows the moment convergence of the estimator under a stationarity. Moreover, our results can yield information criteria…
We have shown in previous work that statistical inference for cooperative sequential adsorption model can be based on maximum likelihood estimation. In this paper we continue this research and establish asymptotic normality of the maximum likelihood estimator in thermodynamic limit. We also perform and discuss some numerical simulations of the model.
We consider the problem of estimating the joint distribution function of the event time and a continuous mark variable based on censored data. More specifically, the event time is subject to current status censoring and the continuous mark is only observed in case inspection takes place after the event time. The nonparametric maximum likelihood estimator (MLE) in this model is known to be inconsistent. We propose and study an alternative likelihood based estimator, maximizing a smoothed log-likelihood, hence called a maximum smoothed likelihood estimator (MSLE). This estimator is shown to be w…
In finite mixtures of location-scale distributions, if there is no constraint or penalty on the parameters, then the maximum likelihood estimator does not exist because the likelihood is unbounded. To avoid this problem, we consider a penalized likelihood, where the penalty is a function of the minimum of the ratios of the scale parameters and the sample size. It is shown that the penalized maximum likelihood estimator is strongly consistent. We also analyze the consistency of a penalized maximum likelihood estimator where the penalty is imposed on the scale parameters themselves.
We consider statistical models driven by Gaussian and non-Gaussian self-similar processes with long memory and we construct maximum likelihood estimators (MLE) for the drift parameter. Our approach is based on the approximation by random walks of the driving noise. We study the asymptotic behavior of the estimators and we give some numerical simulations to illustrate our results.
In soft decoding of data bits, the log-likelihood ratios are evaluated from the estimated data symbols. For proper constellation diagrams such as QPSK or 16-QAM, these data symbols are often estimated using the linear minimum mean square error (LMMSE) estimator. The LMMSE estimator only fulfills the weak Bayesian unbiasedness constraint. Recently, estimators fulfilling the more stringent component-wise conditionally unbiased (CWCU) constraints have been investigated, such as the CWCU LMMSE estimator. In this paper, we prove that the CWCU LMMSE estimates result in the very same log-likelihood r…
The $α$-Brownian bridge, or scaled Brownian bridge, is a generalization of the Brownian bridge with a scaling parameter that determines how strong the force that pulls the process back to 0 is. The bias of the maximum likelihood estimator of the parameter $α$ is derived and a bias-correction that improves the estimator substantially is proposed. The properties of the bias-corrected estimator and four Bayesian estimators based on non-informative priors are evaluated in a simulation study.
In this review, we address the use of Monte Carlo methods for approximating definite integrals of the form $Z = \int L(x) d P(x)$, where $L$ is a target function (often a likelihood) and $P$ a finite measure. We present vertical-likelihood Monte Carlo, which is an approach for designing the importance function $g(x)$ used in importance sampling. Our approach exploits a duality between two random variables: the random draw $X \sim g$, and the corresponding random likelihood ordinate $Y\equiv L(X)$ of the draw. It is natural to specify $g(x)$ and ask: what is the the implied distribution of $Y$?…
