We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically. We develop a generalized empirical likelihood framework---based on distributional uncertainty sets constructed from nonparametric $f$-divergence balls---for Hadamard differentiable functionals, and in particular, stochastic optimization problems. As consequences of this theory, we provide a principled method for choosing the size of distributional uncertainty regions to provide one- and two-sided confidence intervals that achieve exact coverage. We also give an asymptotic expansion for our distributionally robust formulation, showing how robustification regularizes problems by their variance. Finally, we show that optimizers of the distributionally robust formulations we study enjoy (essentially) the same consistency properties as those in classical sample average approximations. Our general approach applies to quickly mixing stationary sequences, including geometrically ergodic Harris recurrent Markov chains.
We propose a two-sample extended empirical likelihood for inference on the difference between two p-dimensional parameters defined by estimating equations. The standard two-sample empirical likelihood for the difference is Bartlett correctable but its domain is a bounded subset of the parameter space. We expand its domain through a composite similarity transformation to derive the two-sample extended empirical likelihood which is defined on the full parameter space. The extended empirical likelihood has the same asymptotic distribution as the standard one and can also achieve the second order …
In this paper, we propose a new statistical inference method for massive data sets, which is very simple and efficient by combining divide-and-conquer method and empirical likelihood. Compared with two popular methods (the bag of little bootstrap and the subsampled double bootstrap), we make full use of data sets, and reduce the computation burden. Extensive numerical studies and real data analysis demonstrate the effectiveness and flexibility of our proposed method. Furthermore, the asymptotic property of our method is derived.
In this paper, we apply empirical likelihood method to inference for the regression parameters in the partial functional linear regression models based on B spline. We prove that the empirical log likelihood ratio for the regression parameters converges in law to a weighted sum of independent chi square distributions and run simulations to assess the finite sample performance of our method.
We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.
Empirical likelihood is a well-known nonparametric method in statistics and has been widely applied in statistical inference. The method has been employed by Lu and Peng (2002) to constructing confidence intervals for the tail index of a heavy-tailed distribution. It is demonstrated in Lu and Peng (2002) that the empirical likelihood-based confidence intervals performs better than confidence intervals based on normal approximation in terms of the coverage probability. In general, the empirical likelihood method can be hindered by its imprecision in the coverage probability when the sample size…
We propose the so-called jackknife empirical likelihood approach for the survey data of general unequal probability sampling designs, and analyze parameters defined according to U-statistics. We prove theoretically that jackknife pseudo-empirical likelihood ratio statistic is asymptotically distributed as a chi-square random variable, and can be used to construct confidence intervals for complex survey samples. In the process of research, we consider with or without auxiliary information, utilizing design weights or calibration weights. Simulation studies are included to examine that in terms …
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
In the present article, we discuss jackknife empirical likelihood (JEL) and adjusted jackknife empirical likelihood (AJEL) based inference for finding confidence intervals for probability weighted moment (PWM). We obtain the asymptotic distribution of the JEL ratio and AJEL ratio statistics. We compare the performance of the proposed confidence intervals with recently developed methods in terms of coverage probability and average length. We also develop JEL and AJEL based test for PWM and study it properties. Finally we illustrate our method using rainfall data of Indian states.
The likelihood function plays a pivotal role in statistical inference; it is adaptable to a wide range of models and the resultant estimators are known to have good properties. However, these results hinge on correct specification of the data generating mechanism. Many modern problems involve extremely complicated distribution functions, which may be difficult -- if not impossible -- to express explicitly. This is a serious barrier to the likelihood approach, which requires not only the specification of a distribution, but the correct distribution. Non-parametric methods are one way to avoid t…
We consider an empirical likelihood inference for parameters defined by general estimating equations when some components of the random observations are subject to missingness. As the nature of the estimating equations is wide-ranging, we propose a nonparametric imputation of the missing values from a kernel estimator of the conditional distribution of the missing variable given the always observable variable. The empirical likelihood is used to construct a profile likelihood for the parameter of interest. We demonstrate that the proposed nonparametric imputation can remove the selection bias …
A Two-step Metropolis Hastings Method for Bayesian Empirical Likelihood Computation with Application to Quantile Regression and Bayesian Model Selection
University of Nebraska-Lincoln（内布拉斯加大学林肯分校）；DFS Group Ltd.（DFS集团有限公司）；Texas A& M University（德克萨斯农工大学）
Empirical likelihood approach is one of non-parametric statistical methods, which is applied to the hypothesis testing or construction of confidence regions for pivotal unknown quantities. This method has been applied to the case of independent identically distributed random variables and second order stationary processes. In recent years, we observe heavy-tailed data in many fields. To model such data suitably, we consider symmetric scalar and multivariate $α$-stable linear processes generated by infinite variance innovation sequence. We use a Whittle likelihood type estimating function in th…
In this paper, we propose an empirical likelihood-based weighted estimator of regression parameter in quantile regression model with nonignorable missing covariates. The proposed estimator is computationally simple and achieves semiparametric efficiency if the probability of missingness on the fully observed variables is correctly specified. The efficiency gain of the proposed estimator over the complete-case-analysis estimator is quantified theoretically and illustrated via simulation and a real data application.
This paper studies the asymptotic properties of and alternative inference methods for kernel density estimation (KDE) for dyadic data. We first establish uniform convergence rates for dyadic KDE. Secondly, we propose a modified jackknife empirical likelihood procedure for inference. The proposed test statistic is asymptotically pivotal regardless of presence of dyadic clustering. The results are further extended to cover the practically relevant case of incomplete dyadic data. Simulations show that this modified jackknife empirical likelihood-based inference procedure delivers precise coverage…
We investigate a generalized empirical likelihood approach in a two-group setting where the constraints on parameters have a form of U-statistics. In this situation, the summands that consist of the constraints for the empirical likelihood are not independent, and a weight of each summand may not have a direct interpretation as a probability point mass, dissimilar to the common empirical likelihood constraints based on independent summands. We show that the resulting empirical likelihood ratio statistic has a weighted chi-squared distribution in the univariate case and a combination of weighte…
We study the empirical likelihood approach to construct confidence intervals for the optimal value and the optimality gap of a given solution, henceforth quantify the statistical uncertainty of sample average approximation, for optimization problems with expected value objectives and constraints where the underlying probability distributions are observed via limited data. This approach relies on two distributionally robust optimization problems posited over the uncertain distribution, with a divergence-based uncertainty set that is suitably calibrated to provide asymptotic statistical guarante…
The authors propose a robust semi-parametric empirical likelihood method to integrate all available information from multiple samples with a common center of measurements. Two different sets of estimating equations are used to improve the classical likelihood inference on the measurement center. The proposed method does not require the knowledge of the functional forms of the probability density functions of related populations. The advantages of the proposed method were demonstrated through the extensive simulation studies by comparing mean squared error, coverage probabilities and average le…
Empirical-likelihood-based confidence intervals for a mean were introduced by Owen [Biometrika 75 (1988) 237-249], where at least a finite second moment is required. This excludes some important distributions, for example, those in the domain of attraction of a stable law with index between 1 and 2. In this article we use a method similar to Qin and Wong [Scand. J. Statist. 23 (1996) 209-219] to derive an empirical-likelihood-based confidence interval for the mean when the underlying distribution has heavy tails. Our method can easily be extended to obtain a confidence interval for any order o…
Change point analysis has become an important research topic in many fields of applications. Several research work has been carried out to detect changes and its locations in time series data. In this paper, a nonparametric method based on the empirical likelihood is proposed to detect the structural changes of the parameters in autoregressive (AR) models . Under certain conditions, the asymptotic null distribution of the empirical likelihood ratio test statistic is proved to be the extreme value distribution. Further, the consistency of the test statistic has been proved. Simulations have bee…
In this paper, we develop statistical inference techniques for the unknown coefficient functions and single-index parameters in single-index varying-coefficient models. We first estimate the nonparametric component via the local linear fitting, then construct an estimated empirical likelihood ratio function and hence obtain a maximum empirical likelihood estimator for the parametric component. Our estimator for parametric component is asymptotically efficient, and the estimator of nonparametric component has an optimal convergence rate. Our results provide ways to construct the confidence regi…
We present simulated standard curves for the calibration of empirical likelihood ratio (ELR) tests of means. With the help of these curves, the nominal significance level of the ELR test can be adjusted in order to achieve (quasi-) exact type I error rate control for a given, finite sample size. By theoretical considerations and by computer simulations, we demonstrate that the adjusted significance level depends most crucially on the skewness and on the kurtosis of the parent distribution. For practical purposes, we tabulate adjusted critical values under several prototypical statistical model…
Motivated by applications to goodness of fit testing, the empirical likelihood approach is generalized to allow for the number of constraints to grow with the sample size and for the constraints to use estimated criteria functions. The latter is needed to deal with nuisance parameters. The proposed empirical likelihood based goodness of fit tests are asymptotically distribution free. For univariate observations, tests for a specified distribution, for a distribution of parametric form, and for a symmetric distribution are presented. For bivariate observations, tests for independence are develo…
Thus far, likelihood-based interval estimates for quantiles have not been studied in the literature on interval censored case 2 data and partly interval censored data, and, in this context, the use of smoothing has not been considered for any type of censored data. This article constructs smoothed weighted empirical likelihood ratio confidence intervals (WELRCI) for quantiles in a unified framework for various types of censored data, including right censored data, doubly censored data, interval censored data and partly interval censored data. The fourth order expansion of the weighted empirica…
The celebrated Bernstein von-Mises theorem ensures that credible regions from Bayesian posterior are well-calibrated when the model is correctly-specified, in the frequentist sense that their coverage probabilities tend to the nominal values as data accrue. However, this conventional Bayesian framework is known to lack robustness when the model is misspecified or only partly specified, such as in quantile regression, risk minimization based supervised/unsupervised learning and robust estimation. To overcome this difficulty, we propose a new Bayesian inferential approach that substitutes the (m…
It is well known that the empirical likelihood ratio confidence region suffers finite sample under-coverage issue, and this severely hampers its application in statistical inferences.} The root cause of this under-coverage is an upper limit imposed by the convex hull of the estimating functions that is used in the construction of the profile empirical likelihood. For i.i.d data, various methods have been proposed to solve this issue by modifying the convex hull, but it is not clear how well these methods perform when the data are no longer independent. In this paper, we propose an adjusted blo…
We study an optimization-based approach to construct statistically accurate confidence intervals for simulation performance measures under nonparametric input uncertainty. This approach computes confidence bounds from simulation runs driven by probability weights defined on the data, which are obtained from solving optimization problems under suitably posited averaged divergence constraints. We illustrate how this approach offers benefits in computational efficiency and finite-sample performance compared to the bootstrap and the delta method. While resembling robust optimization, we explain th…
Composite likelihood inference has gained much popularity thanks to its computational manageability and its theoretical properties. Unfortunately, performing composite likelihood ratio tests is inconvenient because of their awkward asymptotic distribution. There are many proposals for adjusting composite likelihood ratio tests in order to recover an asymptotic chi square distribution, but they all depend on the sensitivity and variability matrices. The same is true for Wald-type and score-type counterparts. In realistic applications sensitivity and variability matrices usually need to be estim…
High-dimensional statistical inference with general estimating equations are challenging and remain less explored. In this paper, we study two problems in the area: confidence set estimation for multiple components of the model parameters, and model specifications test. For the first one, we propose to construct a new set of estimating equations such that the impact from estimating the high-dimensional nuisance parameters becomes asymptotically negligible. The new construction enables us to estimate a valid confidence region by empirical likelihood ratio. For the second one, we propose a test …
Let $X_1, \ldots, X_n$ be some i.i.d. observations from a heavy tailed distribution $F$, i.e. such that the common distribution of the excesses over a high threshold $u_n$ can be approximated by a Generalized Pareto Distribution $G_{γ,σ_n}$ with $γ>0$. This work is devoted to the problem of finding confidence regions for the couple $(γ,σ_n)$ : combining the empirical likelihood methodology with estimation equations (close but not identical to the likelihood equations) introduced by J. Zhang (Australian and New Zealand J. Stat n.49(1), 2007), asymptotically valid confidence regions for $(γ,σ_n)…
This paper presents the asymptotic distributions of a general likelihood-based test statistic, derived using results of Wilks and Wald. The general form of the test statistic incorporates the test statistics and associated asymptotic formulae previously derived by Cowan, Cranmer, Gross and Vitells, which are seen to be special cases of the likelihood-based test statistic described here.
A non parametric method based on the empirical likelihood is proposed for detecting the change in the coefficients of high-dimensional linear model where the number of model variables may increase as the sample size increases. This amounts to testing the null hypothesis of no change against the alternative of one change in the regression coefficients. Based on the theoretical asymptotic behaviour of the empirical likelihood ratio statistic, we propose, for a fixed design, a simpler test statistic, easier to use in practice. The asymptotic normality of the proposed test statistic under the null…
In this article, the weighted empirical likelihood is applied to a general setting of two-sample semiparametric models, which includes biased sampling models and case-control logistic regression models as special cases. For various types of censored data, such as right censored data, doubly censored data, interval censored data and partly interval-censored data, the weighted empirical likelihood-based semiparametric maximum likelihood estimator $(\tildeθ_n,\tilde{F}_n)$ for the underlying parameter $θ_0$ and distribution $F_0$ is derived, and the strong consistency of $(\tildeθ_n,\tilde{F}_n)$…
It is an important task in the literature to check whether a fitted autoregressive moving average (ARMA) model is adequate, while the currently used tests may suffer from the size distortion problem when the underlying autoregressive models have low persistence. To fill this gap, this paper proposes two empirical likelihood-based portmanteau tests. The first one is naive but can serve as a benchmark, and the second is for the case with infinite variance innovations. The asymptotic distributions under the null hypothesis are derived under mild moment conditions, and their usefulness is demonstr…
We propose an empirical likelihood ratio test for nonparametric model selection, where the competing models may be nested, nonnested, overlapping, misspecified, or correctly specified. It compares the squared prediction errors of models based on the cross-validation and allows for heteroscedasticity of the errors of models. We develop its asymptotic distributions for comparing additive models and varying-coefficient models and extend it to test significance of variables in additive models with massive data. The method is applicable to model selection among supervised learning models. To facili…
Linear mixed-effects models are widely used in analyzing repeated measures data, including clustered and longitudinal data, where inferences of both fixed effects and variance components are of importance. Unlike the fixed effect inference that has been well studied, inference on the variance components is more challenging due to null value being on the boundary and the nuisance parameters of the fixed effects. Existing methods often require strong distributional assumptions on the random effects and random errors. In this paper, we develop empirical likelihood-based methods for the inference …
The density ratio model (DRM) provides a flexible and useful platform for combining information from multiple sources. In this paper, we consider statistical inference under two-sample DRMs with additional parameters defined through and/or additional auxiliary information expressed as estimating equations. We examine the asymptotic properties of the maximum empirical likelihood estimators (MELEs) of the unknown parameters in the DRMs and/or defined through estimating equations, and establish the chi-square limiting distributions for the empirical likelihood ratio (ELR) statistics. We show that…
In this work, we study non-parametric hypothesis testing problem with distribution function constraints. The empirical likelihood ratio test has been widely used in testing problems with moment (in)equality constraints. However, some detection problems cannot be described using moment (in)equalities. We propose a distribution function constraint along with an empirical likelihood ratio test. This detector is applicable to a wide variety of robust parametric/non-parametric detection problems. Since the distribution function constraints provide a more exact description of the null hypothesis, th…
This paper formulates a penalized empirical likelihood (PEL) method for inference on the population mean when the dimension of the observations may grow faster than the sample size. Asymptotic distributions of the PEL ratio statistic is derived under different component-wise dependence structures of the observations, namely, (i) non-Ergodic, (ii) long-range dependence and (iii) short-range dependence. It follows that the limit distribution of the proposed PEL ratio statistic can vary widely depending on the correlation structure, and it is typically different from the usual chi-squared limit o…
The area under a receiver operating characteristic curve (AUC) is a useful tool to assess the performance of continuous-scale diagnostic tests on binary classification. In this article, we propose an empirical likelihood (EL) method to construct confidence intervals for the AUC from data collected by ranked set sampling (RSS). The proposed EL-based method enables inferences without assumptions required in existing nonparametric methods and takes advantage of the sampling efficiency of RSS. We show that for both balanced and unbalanced RSS, the EL-based point estimate is the Mann-Whitney statis…
In multi-center clinical trials, due to various reasons, the individual-level data are strictly restricted to be assessed publicly. Instead, the summarized information is widely available from published results. With the advance of computational technology, it has become very common in data analyses to run on hundreds or thousands of machines simultaneous, with the data distributed across those machines and no longer available in a single central location. How to effectively assemble the summarized clinical data information or information from each machine in parallel computation has become a …
This paper considers the maximum generalized empirical likelihood (GEL) estimation and inference on parameters identified by high dimensional moment restrictions with weakly dependent data when the dimensions of the moment restrictions and the parameters diverge along with the sample size. The consistency with rates and the asymptotic normality of the GEL estimator are obtained by properly restricting the growth rates of the dimensions of the parameters and the moment restrictions, as well as the degree of data dependence. It is shown that even in the high dimensional time series setting, the …
In this paper, in order to test whether changes have occurred in a nonlinear parametric regression, we propose a nonparametric method based on the empirical likelihood. Firstly, we test the null hypothesis of no-change against the alternative of one change in the regression parameters. Under null hypothesis, the consistency and the convergence rate of the regression parameter estimators are proved. The asymptotic distribution of the test statistic under the null hypothesis is obtained, which allows to find the asymptotic critical value. On the other hand, we prove that the proposed test statis…
