The problem of estimation of the distribution parameters on the sample when the part of these parameters are discrete (e.g. integer) is considered. We prove that the rate of convergence of MLE estimates under the natural conditions on the distribution density is exponentially fast.
We extend the constrained maximum likelihood estimation theory for parameters of a completely identified model, proposed by Aitchison and Silvey (1958), to parameters arising from a partially identified model. With a partially identified model, some parameters of the model may only be identified through constraints imposed by additional assumptions. We show that, under certain conditions, the constrained maximum likelihood estimator exists and locally maximize the likelihood function subject to constraints. We then study the asymptotic distribution of the estimator and propose a numerical algo…
Growth in both size and complexity of modern data challenges the applicability of traditional likelihood-based inference. Composite likelihood (CL) methods address the difficulties related to model selection and computational intractability of the full likelihood by combining a number of low-dimensional likelihood objects into a single objective function used for inference. This paper introduces a procedure to combine partial likelihood objects from a large set of feasible candidates and simultaneously carry out parameter estimation. The new method constructs estimating equations balancing sta…
The change-plane Cox model is a popular tool for the subgroup analysis of survival data. Despite the rich literature on this model, there has been limited investigation into the asymptotic properties of the estimators of the finite-dimensional parameter. Particularly, the convergence rate, not to mention the asymptotic distribution, has not been fully characterized for the general model where classification is based on multiple covariates. To bridge this theoretical gap, this study proposes a maximum smoothed partial likelihood estimator and establishes the following asymptotic properties. Fir…
In this paper, we apply empirical likelihood method to inference for the regression parameters in the partial functional linear regression models based on B spline. We prove that the empirical log likelihood ratio for the regression parameters converges in law to a weighted sum of independent chi square distributions and run simulations to assess the finite sample performance of our method.
Temporal Point Processes (TPP) with partial likelihoods involving a latent structure often entail an intractable marginalization, thus making inference hard. We propose a novel approach to Maximum Likelihood Estimation (MLE) involving approximate inference over the latent variables by minimizing a tight upper bound on the approximation gap. Given a discrete latent variable $Z$, the proposed approximation reduces inference complexity from $O(|Z|^c)$ to $O(|Z|)$. We use convex conjugates to determine this upper bound in a closed form and show that its addition to the optimization objective resul…
We consider the problem of frequency estimation of the periodic signal multiplied by a stationary Gaussian process (Ornstein-Uhlenbeck) and observed in the presence of the white Gaussian noise. We show the consistency and asymptotic normality of the maximum likelihood estimator in the asymptotics of small noise in observations. The model of observations is a linear nonhomogeneous partially observed system and the construction and study of the estimator is essentialy based on the asymptotics of the equations of Kalman-Bucy filtration.
When are inferences (whether Direct-Likelihood, Bayesian, or Frequentist) obtained from partial data valid? This paper answers this question by offering a new asymptotic theory about inference with missing data that is more general than existing theories. It proves that as the sample size increases and the extent of missingness decreases, the average-loglikelihood function generated by partial data and that ignores the missingness mechanism will converge in probability to that which would have been generated by complete data; and if the data are Missing at Random, this convergence depends only…
Generalized likelihoods are commonly used to obtain consistent estimators with attractive computational and robustness properties. Formally, any generalized likelihood can be used to define a generalized posterior distribution, but an arbitrarily defined "posterior" cannot be expected to appropriately quantify uncertainty in any meaningful sense. In this article, we provide sufficient conditions under which generalized posteriors exhibit concentration, asymptotic normality (Bernstein-von Mises), an asymptotically correct Laplace approximation, and asymptotically correct frequentist coverage. W…
We revisit the problem of estimating the parameters of a partially observed diffusion process, consisting of a hidden state process and an observed process, with a continuous time parameter. The estimation is to be done online, i.e. the parameter estimate should be updated recursively based on the observation filtration. We provide a theoretical analysis of the stochastic gradient ascent algorithm on the incomplete-data log-likelihood. The convergence of the algorithm is proved under suitable conditions regarding the ergodicity of the process consisting of state, filter, and tangent filter. Ad…
George Mason University（乔治梅森大学）；U.S. Census, CSRM（美国人口普查局CSRM）
Recently, there has been significant interest in linear regression in the situation where predictors and responses are not observed in matching pairs corresponding to the same statistical unit as a consequence of separate data collection and uncertainty in data integration. Mismatched pairs can considerably impact the model fit and disrupt the estimation of regression parameters. In this paper, we present a method to adjust for such mismatches under ``partial shuffling" in which a sufficiently large fraction of (predictors, response)-pairs are observed in their correct correspondence. The prop…
Partially observed cured data occur in the analysis of spontaneous abortion (SAB) in observational studies in pregnancy. In contrast to the traditional cured data, such data has an observable `cured' portion as women who do not abort spontaneously. The data is also subject to left truncate in addition to right-censoring because women may enter or withdraw from a study any time during their pregnancy. Left truncation in particular causes unique bias in the presence of a cured portion. In this paper, we study a cure rate model and develop a conditional nonparametric maximum likelihood approach. …
The quasi likelihood analysis is generalized to the partial quasi likelihood analysis. Limit theorems for the quasi likelihood estimators, especially the quasi Bayesian estimator, are derived in the situation where existence of a slow mixing component prohibits the Rosenthal type inequality from applying to the derivation of the polynomial type large deviation inequality for the statistical random field. We give two illustrative examples.
We propose a likelihood ratio statistic for forming hypothesis tests and confidence intervals for a nonparametrically estimated univariate regression function, based on the shape restriction of concavity (alternatively, convexity). Dealing with the likelihood ratio statistic requires studying an estimator satisfying a null hypothesis, that is, studying a concave least-squares estimator satisfying a further equality constraint. We study this null hypothesis least-squares estimator (NLSE) here, and use it to study our likelihood ratio statistic. The NLSE is the solution to a convex program, and …
The parameter estimation of epidemic data-driven models is a crucial task. In some cases, we can formulate a better model by describing uncertainty with appropriate noise terms. However, because of the limited extent and partial information, (in general) this kind of model leads to intractable likelihoods. Here, we illustrate how a stochastic extension of the SEIR model improves the uncertainty quantification of an overestimated MCMC scheme based on its deterministic model to count reported-confirmed COVID-19 cases in Mexico City. Using a particular mechanism to manage missing data, we develop…
We consider the problem of statistical inference for a class of partially-observed diffusion processes, with discretely-observed data and finite-dimensional parameters. We construct unbiased estimators of the score function, i.e. the gradient of the log-likelihood function with respect to parameters, with no time-discretization bias. These estimators can be straightforwardly employed within stochastic gradient methods to perform maximum likelihood estimation or Bayesian inference. As our proposed methodology only requires access to a time-discretization scheme such as the Euler-Maruyama method…
We propose an inferential approach for maximum likelihood estimation of the hidden Markov models for continuous responses. We extend to the case of longitudinal observations the finite mixture model of multivariate Gaussian distributions with Missing At Random (MAR) outcomes, also accounting for possible dropout. The resulting hidden Markov model accounts for different types of missing pattern: (i) partially missing outcomes at a given time occasion; (ii) completely missing outcomes at a given time occasion (intermittent pattern); (iii) dropout before the end of the period of observation (mono…
In this work we introduce the class of beta autoregressive fractionally integrated moving average models for continuous random variables taking values in the continuous unit interval $(0,1)$. The proposed model accommodates a set of regressors and a long-range dependent time series structure. We derive the partial likelihood estimator for the parameters of the proposed model, obtain the associated score vector and Fisher information matrix. We also prove the consistency and asymptotic normality of the estimator under mild conditions. Hypotheses testing, diagnostic tools and forecasting are als…
University College London（伦敦大学学院）；The Alan Turing Institute（艾伦·图灵研究所）
We leverage neural networks as universal approximators of monotonic functions to build a parameterization of conditional cumulative distribution functions (CDFs). By the application of automatic differentiation with respect to response variables and then to parameters of this CDF representation, we are able to build black box CDF and density estimators. A suite of families is introduced as alternative constructions for the multivariate case. At one extreme, the simplest construction is a competitive density estimator against state-of-the-art deep learning methods, although it does not provide …
We propose a new approach to address the text classification problems when learning with partial labels is beneficial. Instead of offering each training sample a set of candidate labels, we assign negative-oriented labels to the ambiguous training examples if they are unlikely fall into certain classes. We construct our new maximum likelihood estimators with self-correction property, and prove that under some conditions, our estimators converge faster. Also we discuss the advantages of applying one of our estimator to a fully supervised learning problem. The proposed method has potential appli…
Approximate Bayesian computation (ABC) and other likelihood-free inference methods have gained popularity in the last decade, as they allow rigorous statistical inference for complex models without analytically tractable likelihood functions. A key component for accurate inference with ABC is the choice of summary statistics, which summarize the information in the data, but at the same time should be low-dimensional for efficiency. Several dimension reduction techniques have been introduced to automatically construct informative and low-dimensional summaries from a possibly large pool of candi…
We consider a problem of parameter estimation for the state space model described by linear stochastic differential equations. We assume that an unobservable Ornstein-Uhlenbeck process drives another observable process by the linear stochastic differential equation, and these two processes depend on some unknown parameters. We construct the quasi-likelihood estimator (QMLE) of the unknown parameters and show asymptotic properties of the estimator.
Analysis of 2 by 2 tables and two-sample survival data has been widely used. Exact calculation is computational intractable for conditional likelihood inference in odds ratio models with large marginals in 2 by 2 tables, or partial likelihood inference in Cox's proportional hazards models with considerable tied event times. Approximate methods are often employed, but their statistical properties have not been formally studied while taking into account the approximation involved. We develop new methods and theory by constructing suitable estimating functions while leveraging knowledge from cond…
