The goal of this paper is to describe the application of quasi-likelihood estimating equations for spatially correlated binary data. In this paper, a logistic function is used to model the marginal probability of binary responses in terms of parameters of interest. With mild assumptions on the correlations, the Leonov-Shiryaev formula combined with a comparison of characteristic functions can be used to establish asymptotic normality for linear combinations of the binary responses. The consistency and asymptotic normality for quasi-likelihood estimates can then be derived. By modeling spatial correlation with a variogram, we apply these asymptotic results to test independence of two spatially correlated binary outcomes and illustrate the concepts with a well-known example based on data from Lansing Woods. The comparison of generalized estimating equations and the proposed approach is also discussed.
We propose a two-sample extended empirical likelihood for inference on the difference between two p-dimensional parameters defined by estimating equations. The standard two-sample empirical likelihood for the difference is Bartlett correctable but its domain is a bounded subset of the parameter space. We expand its domain through a composite similarity transformation to derive the two-sample extended empirical likelihood which is defined on the full parameter space. The extended empirical likelihood has the same asymptotic distribution as the standard one and can also achieve the second order …
We consider an empirical likelihood inference for parameters defined by general estimating equations when some components of the random observations are subject to missingness. As the nature of the estimating equations is wide-ranging, we propose a nonparametric imputation of the missing values from a kernel estimator of the conditional distribution of the missing variable given the always observable variable. The empirical likelihood is used to construct a profile likelihood for the parameter of interest. We demonstrate that the proposed nonparametric imputation can remove the selection bias …
The density ratio model (DRM) provides a flexible and useful platform for combining information from multiple sources. In this paper, we consider statistical inference under two-sample DRMs with additional parameters defined through and/or additional auxiliary information expressed as estimating equations. We examine the asymptotic properties of the maximum empirical likelihood estimators (MELEs) of the unknown parameters in the DRMs and/or defined through estimating equations, and establish the chi-square limiting distributions for the empirical likelihood ratio (ELR) statistics. We show that…
We consider the estimation of the affine parameter (and power-law exponent) in the preferential attachment model with random initial degrees. We derive the likelihood, and show that the maximum likelihood estimator (MLE) is asymptotically normal and efficient. We also propose a quasi-maximum-likelihood estimator (QMLE) to overcome the MLE's dependence on the history of the initial degrees. To demonstrate the power of our idea, we present numerical simulations.
Strong consistency of the quasi-maximum likelihood estimator is given for a general class of multidimensional causal processes based on asyMmetric laplacian innovation.
We consider nonsynchronous sampling of parameterized stochastic regression models, which contain stochastic differential equations. Constructing a quasi-likelihood function, we prove that the quasi-maximum likelihood estimator and the Bayes type estimator are consistent and asymptotically mixed normal when the sampling frequency of the nonsynchronous data becomes large.
In many families of distributions, maximum likelihood estimation is intractable because the normalization constant for the density which enters into the likelihood function is not easily available. The score matching estimator of Hyvärinen (2005) provides an alternative where this normalization constant is not required. The corresponding estimating equations become linear for an exponential family. The score matching estimator is shown to be consistent and asymptotically normally distributed for such models, although not necessarily efficient. Gaussian linear concentration models are examples …
We develop and analyze $M$-estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a non-asymptotic variational characterization of $f$-divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The resulting estimators are simple to implement, requiring only the solution of standard convex programs. We present an analysis of consistency and convergence for these estimators. Given conditions only on the ratios of densities, we show that our estimators can achieve …
An approximate maximum likelihood method of estimation of diffusion parameters $(\vartheta,σ)$ based on discrete observations of a diffusion $X$ along fixed time-interval $[0,T]$ and Euler approximation of integrals is analyzed. We assume that $X$ satisfies a SDE of form $dX_t =μ(X_t ,\vartheta )\, dt+\sqrtσ b(X_t )\, dW_t$, with non-random initial condition. SDE is nonlinear in $\vartheta$ generally. Based on assumption that maximum likelihood estimator $\hat{\vartheta}_T$ of the drift parameter based on continuous observation of a path over $[0,T]$ exists we prove that measurable estimator $…
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
Independent Component Analysis (ICA) models are very popular semiparametric models in which we observe independent copies of a random vector $X = AS$, where $A$ is a non-singular matrix and $S$ has independent components. We propose a new way of estimating the unmixing matrix $W = A^{-1}$ and the marginal distributions of the components of $S$ using nonparametric maximum likelihood. Specifically, we study the projection of the empirical distribution onto the subset of ICA distributions having log-concave marginals. We show that, from the point of view of estimating the unmixing matrix, it make…
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
This paper establishes the almost sure convergence and asymptotic normality of levels and differenced quasi maximum-likelihood (QML) estimators of dynamic panel data models. The QML estimators are robust with respect to initial conditions, conditional and time-series heteroskedasticity, and misspecification of the log-likelihood. The paper also provides an ECME algorithm for calculating levels QML estimates. Finally, it uses Monte Carlo experiments to compare the finite sample performance of levels and differenced QML estimators, the differenced GMM estimator, and the system GMM estimator. In …
We address estimation of parametric coefficients of a pure-jump Lévy driven univariate stochastic differential equation (SDE) model, which is observed at high frequency over a fixed time period. It is known from the previous study Masuda (2013) that adopting the conventional Gaussian quasi-maximum likelihood estimator then leads to an inconsistent estimator. In this paper, under the assumption that the driving Lévy process is locally stable, we extend the Gaussian framework into a non-Gaussian counterpart, by introducing a novel quasi-likelihood function formally based on the small-time stable…
Consider an estimation of the Hurst parameter $H\in(0,1)$ and the volatility parameter $σ>0$ for a fractional Brownian motion with a drift term under high-frequency observations with a finite time interval. In the present paper, we propose a consistent estimator of the parameter $θ=(H,σ)$ combining the ideas of a quasi-likelihood function based on a local Gaussian approximation of a high-frequently observed time series and its frequency-domain approximation. Moreover, we prove an asymptotic normality property of the proposed estimator for all $H\in(0,1)$ when the drift process is constant.
Some quasi-arithmetic means of random variables easily give unbiased strongly consistent closed-form estimators of the joint of the location and scale parameters of the Cauchy distribution. The one-step estimators of those quasi-arithmetic means of the Cauchy distribution are considered. We establish the Bahadur efficiency of the maximum likelihood estimator and the one-step estimators. We also show that the rate of the convergence of the mean-squared errors achieves the Cramer-Rao bound. Our results are also applicable to the circular Cauchy distribution.
This paper investigates the asymptotic theory of the quasi-maximum exponential likelihood estimators (QMELE) for ARMA--GARCH models. Under only a fractional moment condition, the strong consistency and the asymptotic normality of the global self-weighted QMELE are obtained. Based on this self-weighted QMELE, the local QMELE is showed to be asymptotically normal for the ARMA model with GARCH (finite variance) and IGARCH errors. A formal comparison of two estimators is given for some cases. A simulation study is carried out to assess the performance of these estimators, and a real example on the…
We propose a class of estimators for the parameters of a GARCH(p,q) sequence. We show that our estimators are consistent and asymptotically normal under mild conditions. The quasi-maximum likelihood and the likelihood estimators are discussed in detail. We show that the maximum likelihood estimator is optimal. If the tail of the distribution of the innovations is polynomial, even a quasi-maximum likelihood estimator based on exponential density performs better than the standard normal density-based quasi-likelihood estimator of Lee and Hansen and Lumsdaine.
Statistical models incorporating change points are common in practice, especially in the area of biomedicine. This approach is appealing in that a specific parameter is introduced to account for the abrupt change in the response variable relating to a particular independent variable of interest. The statistical challenge one encounters is that the likelihood function is not differentiable with respect to this change point parameter. Consequently, the conventional asymptotic properties for the maximum likelihood estimators fail to hold in this situation. In this paper, we propose an estimating …
This paper considers the empirical likelihood (EL) construction of confidence intervals for a linear functional based on right censored lifetime data. Many of the results in literature show that log EL has a limiting scaled chi-square distribution, where the scale parameter is a function of the unknown asymptotic variance. The scale parameter has to be estimated for the construction. Additional estimation would reduce the coverage accuracy for the parameter. This diminishes a main advantage of the EL method for censored data. By utilizing certain influence functions in an estimating equation, …
Generalized linear models and the quasi-likelihood method extend the ordinary regression models to accommodate more general conditional distributions of the response. Nonparametric methods need no explicit parametric specification, and the resulting model is completely determined by the data themselves. However, nonparametric estimation schemes generally have a slower convergence rate such as the local polynomial smoothing estimation of nonparametric generalized linear models studied in Fan, Heckman and Wand [J. Amer. Statist. Assoc. 90 (1995) 141--150]. In this work, we propose a unified fami…
This paper deals with nonparametric maximum likelihood estimation for Gaussian locally stationary processes. Our nonparametric MLE is constructed by minimizing a frequency domain likelihood over a class of functions. The asymptotic behavior of the resulting estimator is studied. The results depend on the richness of the class of functions. Both sieve estimation and global estimation are considered. Our results apply, in particular, to estimation under shape constraints. As an example, autoregressive model fitting with a monotonic variance function is discussed in detail, including algorithmic …
Statistical inference for discrete time observations of an affine stochastic delay differential equation is considered. The main focus is on maximum pseudo-likelihood estimators, which are easy to calculate in practice. A more general class of prediction-based estimating functions is investigated as well. In particular, the optimal prediction-based estimating function and the asymptotic properties of the estimators are derived. The maximum pseudo-likelihood estimator is a particular case, and an expression is found for the efficiency loss when using the maximum pseudo-likelihood estimator, rat…
In the present work, we have investigated the problem of estimating parameters of several exponential distributions with ordered scale parameters under the linex loss function. We have considered estimating ordered scale parameters when the location parameters are known and unknown. For every case, we consider a class of equivariant estimators, and sufficient condition is obtained under which this class of estimators improves upon the usual estimator. Using this result, we have shown that the restricted maximum likelihood estimator is inadmissible. Finally, for every case, we conduct a simulat…
This paper investigates the Gaussian quasi-likelihood estimation of an exponentially ergodic multidimensional Markov process, which is expressed as a solution to a Lévy driven stochastic differential equation whose coefficients are known except for the finite-dimensional parameters to be estimated, where the diffusion coefficient may be degenerate or even null. We suppose that the process is discretely observed under the rapidly increasing experimental design with step size $h_n$. By means of the polynomial-type large deviation inequality, convergence of the corresponding statistical random fi…
We extend the constrained maximum likelihood estimation theory for parameters of a completely identified model, proposed by Aitchison and Silvey (1958), to parameters arising from a partially identified model. With a partially identified model, some parameters of the model may only be identified through constraints imposed by additional assumptions. We show that, under certain conditions, the constrained maximum likelihood estimator exists and locally maximize the likelihood function subject to constraints. We then study the asymptotic distribution of the estimator and propose a numerical algo…
The non-Gaussian quasi maximum likelihood estimator is frequently used in GARCH models with intension to improve the efficiency of the GARCH parameters. However, unless the quasi-likelihood happens to be the true one, non-Gaussian QMLE methods suffers inconsistency even if shape parameters in the quasi-likelihood are estimated. To correct this bias, we identify an unknown scale parameter that is critical to the consistent estimation of non-Gaussian QMLE, and propose a two-step non-Gaussian QMLE (2SNG-QMLE) for estimation of the scale parameter and GARCH parameters. This novel approach is consi…
This paper considers an empirical likelihood inference for parameters defined by general estimating equations, when data are missing at random. The efficiency of existing estimators depends critically on correctly specifying the conditional expectation of the estimating function given the observed components of the random observations. When the conditional expectation is not correctly specified, the efficiency of estimation can be severely compromised even if the propensity function (of missingness) is correctly specified. We propose an efficient estimator which enjoys the double-robustness pr…
We classify the two-way independence quasi-independence models (or independence models with structural zeros) that have rational maximum likelihood estimators, or MLEs. We give a necessary and sufficient condition on the bipartite graph associated to the model for the MLE to be rational. In this case, we give an explicit formula for the MLE in terms of combinatorial features of this graph. We also use the Horn uniformization to show that for general log-linear models $\mathcal{M}$ with rational MLE, any model obtained by restricting to a face of the cone of sufficient statistics of $\mathcal{M…
This paper develops a particle filter maximum likelihood estimator for the competitive storage model. The estimator is suitable for inference problems in commodity markets where only reliable price data is available for estimation, and shocks are temporally dependent. The estimator efficiently utilizes the information present in the conditional distribution of prices when shocks are not iid. Compared to Deaton and Laroque's composite quasi-maximum likelihood estimator, simulation experiments and real-data estimation show substantial improvements in both bias and precision. Simulation experimen…
An Accurate Standard Error Estimation for Quadratic Exponential Logistic Regressions by Applying Generalized Estimating Equations to Pseudo-Likelihoods
This paper establishes the strong consistency and asymptotic normality of the quasi-maximum likelihood estimator (QMLE) for a GARCH process with periodically time-varying parameters. We first give a necessary and sufficient condition for the existence of a strictly periodically stationary solution for the periodic GARCH (P-GARCH) equation. As a result, it is shown that the moment of some positive order of the P-GARCH solution is finite, under which we prove the strong consistency and asymptotic normality (CAN) of the QMLE without any condition on the moments of the underlying process.
In this paper, we present a maximum likelihood method for estimating the parameters of a univariate Hawkes process with self-excitation or inhibition. Our work generalizes techniques and results that were restricted to the self-exciting scenario. The proposed estimator is implemented for the classical exponential kernel and we show that, in the inhibition context, our procedure provides more accurate estimations than current alternative approaches.
Given p independent normal populations, we consider the problem of estimating the mean of those populations, that based on the observed data, give the strongest signals. We explicitly condition on the ranking of the sample means, and consider a constrained conditional maximum likelihood (CCMLE) approach, avoiding the use of any priors and of any sparsity requirement between the population means. Our results show that if the observed means are too close together, we should in fact use the grand mean to estimate the mean of the population with the larger sample mean. If they are separated by mor…
The problem of estimating the directed information rate between two discrete processes $\{X_n\}$ and $\{Y_n\}$ via the plug-in (or maximum-likelihood) estimator is considered. When the joint process $\{(X_n,Y_n)\}$ is a Markov chain of a given memory length, the plug-in estimator is shown to be asymptotically Gaussian and to converge at the optimal rate $O(1/\sqrt{n})$ under appropriate conditions; this is the first estimator that has been shown to achieve this rate. An important connection is drawn between the problem of estimating the directed information rate and that of performing a hypoth…
We consider quasi maximum likelihood (QML) estimation for general non-Gaussian discrete-ime linear state space models and equidistantly observed multivariate Lévy-driven continuoustime autoregressive moving average (MCARMA) processes. In the discrete-time setting, we prove strong consistency and asymptotic normality of the QML estimator under standard moment assumptions and a strong-mixing condition on the output process of the state space model. In the second part of the paper, we investigate probabilistic and analytical properties of equidistantly sampled continuous-time state space models a…
The paper describes a new class of capture-recapture models for closed populations when individual covariates are available. The novelty consists in combining a latent class model for the distribution of the capture history, where the class weights and the conditional distributions given the latent may depend on covariates, with a model for the marginal distribution of the available covariates as in \cite{Liu2017}. In addition, any general form of serial dependence is allowed when modeling capture histories conditionally on the latent and covariates. A Fisher-scoring algorithm for maximum like…
Maximum likelihood estimation is a common method of estimating the parameters of the probability distribution from a given sample. This paper aims to introduce the maximum likelihood estimation in the framework of sublinear expectation. We find the maximum likelihood estimator for the parameters of the maximal distribution via the solution of the associated minimax problem, which coincides with the optimal unbiased estimation given by Jin and Peng \cite{JP21}. A general estimation method for samples with dependent structure is also provided. This result provides a theoretical foundation for th…
Neural networks (NN) play a central role in modern Artificial intelligence (AI) technology and has been successfully used in areas such as natural language processing and image recognition. While majority of NN applications focus on prediction and classification, there are increasing interests in studying statistical inference of neural networks. The study of NN statistical inference can enhance our understanding of NN statistical proprieties. Moreover, it can facilitate the NN-based hypothesis testing that can be applied to hypothesis-driven clinical and biomedical research. In this paper, we…
High-dimensional statistical inference with general estimating equations are challenging and remain less explored. In this paper, we study two problems in the area: confidence set estimation for multiple components of the model parameters, and model specifications test. For the first one, we propose to construct a new set of estimating equations such that the impact from estimating the high-dimensional nuisance parameters becomes asymptotically negligible. The new construction enables us to estimate a valid confidence region by empirical likelihood ratio. For the second one, we propose a test …
The notion that an independent central bank reduces a country's inflation is a controversial hypothesis. To date, it has not been possible to satisfactorily answer this question because the complex macroeconomic structure that gives rise to the data has not been adequately incorporated into statistical analyses. We develop a causal model that summarizes the economic process of inflation. Based on this causal model and recent data, we discuss and identify the assumptions under which the effect of central bank independence on inflation can be identified and estimated. Given these and alternative…
This paper concerns the parameter estimation problem for the quadratic potential energy in interacting particle systems from continuous-time and single-trajectory data. Even though such dynamical systems are high-dimensional, we show that the vanilla maximum likelihood estimator (without regularization) is able to estimate the interaction potential parameter with optimal rate of convergence simultaneously in mean-field limit and in long-time dynamics. This to some extend avoids the curse-of-dimensionality for estimating large dynamical systems under symmetry of the particle interaction.
A number of studies have suggested using comparisons between DNA sequences of closely related bacterial isolates to estimate the relative rate of recombination to mutation for that bacterial species. We consider such an approach which uses single-locus variants: pairs of isolates whose DNA differ at a single gene locus. One way of deriving point estimates for the relative rate of recombination to mutation from such data is to use composite likelihood methods. We extend recent work in this area so as to be able to construct confidence intervals for our estimates, without needing to resort to co…
