We propose an empirical likelihood ratio test for nonparametric model selection, where the competing models may be nested, nonnested, overlapping, misspecified, or correctly specified. It compares the squared prediction errors of models based on the cross-validation and allows for heteroscedasticity of the errors of models. We develop its asymptotic distributions for comparing additive models and varying-coefficient models and extend it to test significance of variables in additive models with massive data. The method is applicable to model selection among supervised learning models. To facilitate implementation of the test, we provide a fast calculation procedure. Simulations show that the proposed tests work well and have favorable finite sample performance over some existing approaches. The methodology is validated on an empirical application.
We address the issue of performing inference on the parameters that index a bimodal extension of the Birnbaum-Saunders distribution (BS). We show that maximum likelihood point estimation can be problematic since the standard nonlinear optimization algorithms may fail to converge. To deal with this problem, we penalize the log-likelihood function. The numerical evidence we present show that maximum likelihood estimation based on such penalized function is made considerably more reliable. We also consider hypothesis testing inference based on the penalized log-likelihood function. In particular,…
In this paper, we apply empirical likelihood method to inference for the regression parameters in the partial functional linear regression models based on B spline. We prove that the empirical log likelihood ratio for the regression parameters converges in law to a weighted sum of independent chi square distributions and run simulations to assess the finite sample performance of our method.
We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.
We propose the so-called jackknife empirical likelihood approach for the survey data of general unequal probability sampling designs, and analyze parameters defined according to U-statistics. We prove theoretically that jackknife pseudo-empirical likelihood ratio statistic is asymptotically distributed as a chi-square random variable, and can be used to construct confidence intervals for complex survey samples. In the process of research, we consider with or without auxiliary information, utilizing design weights or calibration weights. Simulation studies are included to examine that in terms …
In the present article, we discuss jackknife empirical likelihood (JEL) and adjusted jackknife empirical likelihood (AJEL) based inference for finding confidence intervals for probability weighted moment (PWM). We obtain the asymptotic distribution of the JEL ratio and AJEL ratio statistics. We compare the performance of the proposed confidence intervals with recently developed methods in terms of coverage probability and average length. We also develop JEL and AJEL based test for PWM and study it properties. Finally we illustrate our method using rainfall data of Indian states.
In this paper, we propose an empirical likelihood-based weighted estimator of regression parameter in quantile regression model with nonignorable missing covariates. The proposed estimator is computationally simple and achieves semiparametric efficiency if the probability of missingness on the fully observed variables is correctly specified. The efficiency gain of the proposed estimator over the complete-case-analysis estimator is quantified theoretically and illustrated via simulation and a real data application.
We investigate a generalized empirical likelihood approach in a two-group setting where the constraints on parameters have a form of U-statistics. In this situation, the summands that consist of the constraints for the empirical likelihood are not independent, and a weight of each summand may not have a direct interpretation as a probability point mass, dissimilar to the common empirical likelihood constraints based on independent summands. We show that the resulting empirical likelihood ratio statistic has a weighted chi-squared distribution in the univariate case and a combination of weighte…
The authors propose a robust semi-parametric empirical likelihood method to integrate all available information from multiple samples with a common center of measurements. Two different sets of estimating equations are used to improve the classical likelihood inference on the measurement center. The proposed method does not require the knowledge of the functional forms of the probability density functions of related populations. The advantages of the proposed method were demonstrated through the extensive simulation studies by comparing mean squared error, coverage probabilities and average le…
Change point analysis has become an important research topic in many fields of applications. Several research work has been carried out to detect changes and its locations in time series data. In this paper, a nonparametric method based on the empirical likelihood is proposed to detect the structural changes of the parameters in autoregressive (AR) models . Under certain conditions, the asymptotic null distribution of the empirical likelihood ratio test statistic is proved to be the extreme value distribution. Further, the consistency of the test statistic has been proved. Simulations have bee…
We present simulated standard curves for the calibration of empirical likelihood ratio (ELR) tests of means. With the help of these curves, the nominal significance level of the ELR test can be adjusted in order to achieve (quasi-) exact type I error rate control for a given, finite sample size. By theoretical considerations and by computer simulations, we demonstrate that the adjusted significance level depends most crucially on the skewness and on the kurtosis of the parent distribution. For practical purposes, we tabulate adjusted critical values under several prototypical statistical model…
The celebrated Bernstein von-Mises theorem ensures that credible regions from Bayesian posterior are well-calibrated when the model is correctly-specified, in the frequentist sense that their coverage probabilities tend to the nominal values as data accrue. However, this conventional Bayesian framework is known to lack robustness when the model is misspecified or only partly specified, such as in quantile regression, risk minimization based supervised/unsupervised learning and robust estimation. To overcome this difficulty, we propose a new Bayesian inferential approach that substitutes the (m…
It is well known that the empirical likelihood ratio confidence region suffers finite sample under-coverage issue, and this severely hampers its application in statistical inferences.} The root cause of this under-coverage is an upper limit imposed by the convex hull of the estimating functions that is used in the construction of the profile empirical likelihood. For i.i.d data, various methods have been proposed to solve this issue by modifying the convex hull, but it is not clear how well these methods perform when the data are no longer independent. In this paper, we propose an adjusted blo…
We study an optimization-based approach to construct statistically accurate confidence intervals for simulation performance measures under nonparametric input uncertainty. This approach computes confidence bounds from simulation runs driven by probability weights defined on the data, which are obtained from solving optimization problems under suitably posited averaged divergence constraints. We illustrate how this approach offers benefits in computational efficiency and finite-sample performance compared to the bootstrap and the delta method. While resembling robust optimization, we explain th…
High-dimensional statistical inference with general estimating equations are challenging and remain less explored. In this paper, we study two problems in the area: confidence set estimation for multiple components of the model parameters, and model specifications test. For the first one, we propose to construct a new set of estimating equations such that the impact from estimating the high-dimensional nuisance parameters becomes asymptotically negligible. The new construction enables us to estimate a valid confidence region by empirical likelihood ratio. For the second one, we propose a test …
It is an important task in the literature to check whether a fitted autoregressive moving average (ARMA) model is adequate, while the currently used tests may suffer from the size distortion problem when the underlying autoregressive models have low persistence. To fill this gap, this paper proposes two empirical likelihood-based portmanteau tests. The first one is naive but can serve as a benchmark, and the second is for the case with infinite variance innovations. The asymptotic distributions under the null hypothesis are derived under mild moment conditions, and their usefulness is demonstr…
Linear mixed-effects models are widely used in analyzing repeated measures data, including clustered and longitudinal data, where inferences of both fixed effects and variance components are of importance. Unlike the fixed effect inference that has been well studied, inference on the variance components is more challenging due to null value being on the boundary and the nuisance parameters of the fixed effects. Existing methods often require strong distributional assumptions on the random effects and random errors. In this paper, we develop empirical likelihood-based methods for the inference …
The area under a receiver operating characteristic curve (AUC) is a useful tool to assess the performance of continuous-scale diagnostic tests on binary classification. In this article, we propose an empirical likelihood (EL) method to construct confidence intervals for the AUC from data collected by ranked set sampling (RSS). The proposed EL-based method enables inferences without assumptions required in existing nonparametric methods and takes advantage of the sampling efficiency of RSS. We show that for both balanced and unbalanced RSS, the EL-based point estimate is the Mann-Whitney statis…
In multi-center clinical trials, due to various reasons, the individual-level data are strictly restricted to be assessed publicly. Instead, the summarized information is widely available from published results. With the advance of computational technology, it has become very common in data analyses to run on hundreds or thousands of machines simultaneous, with the data distributed across those machines and no longer available in a single central location. How to effectively assemble the summarized clinical data information or information from each machine in parallel computation has become a …
The Gini correlation plays an important role in measuring dependence of random variables with heavy tailed distributions, whose properties are a mixture of Pearson's and Spearman's correlations. Due to the structure of this dependence measure, there are two Gini correlations between each pair of random variables, which are not equal in general. Both the Gini correlation and the equality of the two Gini correlations play important roles in Economics. In the literature, there are limited papers focusing on the inference of the Gini correlations and their equality testing. In this paper, we devel…
Nonresponse is common in surveys. When the response probability of a survey variable $Y$ depends on $Y$ through an observed auxiliary categorical variable $Z$ (i.e., the response probability of $Y$ is conditionally independent of $Y$ given $Z$), a simple method often used in practice is to use $Z$ categories as imputation cells and construct estimators by imputing nonrespondents or reweighting respondents within each imputation cell. This simple method, however, is inefficient when some $Z$ categories have small sizes and ad hoc methods are often applied to collapse small imputation cells. Ass…
Nonparametric empirical Bayes methods provide a flexible and attractive approach to high-dimensional data analysis. One particularly elegant empirical Bayes methodology, involving the Kiefer-Wolfowitz nonparametric maximum likelihood estimator (NPMLE) for mixture models, has been known for decades. However, implementation and theoretical analysis of the Kiefer-Wolfowitz NPMLE are notoriously difficult. A fast algorithm was recently proposed that makes NPMLE-based procedures feasible for use in large-scale problems, but the algorithm calculates only an approximation to the NPMLE. In this paper …
Empirical likelihood is a well-known nonparametric method in statistics and has been widely applied in statistical inference. The method has been employed by Lu and Peng (2002) to constructing confidence intervals for the tail index of a heavy-tailed distribution. It is demonstrated in Lu and Peng (2002) that the empirical likelihood-based confidence intervals performs better than confidence intervals based on normal approximation in terms of the coverage probability. In general, the empirical likelihood method can be hindered by its imprecision in the coverage probability when the sample size…
In health-related experiments, treatment effects can be identified using paired data that consist of pre- and post-treatment measurements. In this framework, sequential testing strategies are widely accepted statistical tools in practice. Since performances of parametric sequential testing procedures vitally depend on the validity of the parametric assumptions regarding underlying data distributions, we focus on distribution-free mechanisms for sequentially evaluating treatment effects. In fixed sample size designs, the density-based empirical likelihood (DBEL) methods provide powerful nonpara…
Inverse probability weighting (IPW) is widely used in many areas when data are subject to unrepresentativeness, missingness, or selection bias. An inevitable challenge with the use of IPW is that the IPW estimator can be remarkably unstable if some probabilities are very close to zero. To overcome this problem, at least three remedies have been developed in the literature: stabilizing, thresholding, and trimming. However the final estimators are still IPW type estimators, and inevitably inherit certain weaknesses of the naive IPW estimator: they may still be unstable or biased. We propose a bi…
We study the empirical likelihood approach to construct confidence intervals for the optimal value and the optimality gap of a given solution, henceforth quantify the statistical uncertainty of sample average approximation, for optimization problems with expected value objectives and constraints where the underlying probability distributions are observed via limited data. This approach relies on two distributionally robust optimization problems posited over the uncertain distribution, with a divergence-based uncertainty set that is suitably calibrated to provide asymptotic statistical guarante…
In this paper, we propose a new statistical inference method for massive data sets, which is very simple and efficient by combining divide-and-conquer method and empirical likelihood. Compared with two popular methods (the bag of little bootstrap and the subsampled double bootstrap), we make full use of data sets, and reduce the computation burden. Extensive numerical studies and real data analysis demonstrate the effectiveness and flexibility of our proposed method. Furthermore, the asymptotic property of our method is derived.
We propose a two-sample extended empirical likelihood for inference on the difference between two p-dimensional parameters defined by estimating equations. The standard two-sample empirical likelihood for the difference is Bartlett correctable but its domain is a bounded subset of the parameter space. We expand its domain through a composite similarity transformation to derive the two-sample extended empirical likelihood which is defined on the full parameter space. The extended empirical likelihood has the same asymptotic distribution as the standard one and can also achieve the second order …
A non parametric method based on the empirical likelihood is proposed for detecting the change in the coefficients of high-dimensional linear model where the number of model variables may increase as the sample size increases. This amounts to testing the null hypothesis of no change against the alternative of one change in the regression coefficients. Based on the theoretical asymptotic behaviour of the empirical likelihood ratio statistic, we propose, for a fixed design, a simpler test statistic, easier to use in practice. The asymptotic normality of the proposed test statistic under the null…
During the last decade Levy processes with jumps have received increasing popularity for modelling market behaviour for both derviative pricing and risk management purposes. Chan et al. (2009) introduced the use of empirical likelihood methods to estimate the parameters of various diffusion processes via their characteristic functions which are readily avaiable in most cases. Return series from the market are used for estimation. In addition to the return series, there are many derivatives actively traded in the market whose prices also contain information about parameters of the underlying pr…
The challenges posed by complex stochastic models used in computational ecology, biology and genetics have stimulated the development of approximate approaches to statistical inference. Here we focus on Synthetic Likelihood (SL), a procedure that reduces the observed and simulated data to a set of summary statistics, and quantifies the discrepancy between them through a synthetic likelihood function. SL requires little tuning, but it relies on the approximate normality of the summary statistics. We relax this assumption by proposing a novel, more flexible, density estimator: the Extended Empir…
The density ratio model (DRM) provides a flexible and useful platform for combining information from multiple sources. In this paper, we consider statistical inference under two-sample DRMs with additional parameters defined through and/or additional auxiliary information expressed as estimating equations. We examine the asymptotic properties of the maximum empirical likelihood estimators (MELEs) of the unknown parameters in the DRMs and/or defined through estimating equations, and establish the chi-square limiting distributions for the empirical likelihood ratio (ELR) statistics. We show that…
A Two-step Metropolis Hastings Method for Bayesian Empirical Likelihood Computation with Application to Quantile Regression and Bayesian Model Selection
University of Nebraska-Lincoln（内布拉斯加大学林肯分校）；DFS Group Ltd.（DFS集团有限公司）；Texas A& M University（德克萨斯农工大学）
This paper proposes a local representation for Empirical Likelihood (EL). EL admits the classical local linear quadratic representation by its likelihood ratio property. A local estimator is derived by using the new representation. Consistency, local asymptotic normality, and asymptotic optimality results hold for the new estimator. In particular, when the regularity conditions do not include any differentiability assumption, these asymptotic results are still valid for the local estimator. Simulations illustrate that the local method improves the inference accuracy of EL.
The coefficient of determination, known as $R^2$, is commonly used as a goodness-of-fit criterion for fitting linear models. $R^2$ is somewhat controversial when fitting nonlinear models, although it may be generalised on a case-by-case basis to deal with specific models such as the logistic model. Assume we are fitting a parametric distribution to a data set using, say, the maximum likelihood estimation method. A general approach to measure the goodness-of-fit of the fitted parameters, which is advocated herein, is to use a nonparametric measure for comparison between the empirical distributi…
The likelihood function plays a pivotal role in statistical inference; it is adaptable to a wide range of models and the resultant estimators are known to have good properties. However, these results hinge on correct specification of the data generating mechanism. Many modern problems involve extremely complicated distribution functions, which may be difficult -- if not impossible -- to express explicitly. This is a serious barrier to the likelihood approach, which requires not only the specification of a distribution, but the correct distribution. Non-parametric methods are one way to avoid t…
We consider an empirical likelihood inference for parameters defined by general estimating equations when some components of the random observations are subject to missingness. As the nature of the estimating equations is wide-ranging, we propose a nonparametric imputation of the missing values from a kernel estimator of the conditional distribution of the missing variable given the always observable variable. The empirical likelihood is used to construct a profile likelihood for the parameter of interest. We demonstrate that the proposed nonparametric imputation can remove the selection bias …
Empirical likelihood approach is one of non-parametric statistical methods, which is applied to the hypothesis testing or construction of confidence regions for pivotal unknown quantities. This method has been applied to the case of independent identically distributed random variables and second order stationary processes. In recent years, we observe heavy-tailed data in many fields. To model such data suitably, we consider symmetric scalar and multivariate $α$-stable linear processes generated by infinite variance innovation sequence. We use a Whittle likelihood type estimating function in th…
This paper studies the asymptotic properties of and alternative inference methods for kernel density estimation (KDE) for dyadic data. We first establish uniform convergence rates for dyadic KDE. Secondly, we propose a modified jackknife empirical likelihood procedure for inference. The proposed test statistic is asymptotically pivotal regardless of presence of dyadic clustering. The results are further extended to cover the practically relevant case of incomplete dyadic data. Simulations show that this modified jackknife empirical likelihood-based inference procedure delivers precise coverage…
The main purpose of this paper is to introduce first a new family of empirical test statistics for testing a simple null hypothesis when the vector of parameters of interest are defined through a specific set of unbiased estimating functions. This family of test statistics is based on a distance between two probability vectors, with the first probability vector obtained by maximizing the empirical likelihood on the vector of parameters, and the second vector defined from the fixed vector of parameters under the simple null hypothesis. The distance considered for this purpose is the phi-diverge…
Empirical-likelihood-based confidence intervals for a mean were introduced by Owen [Biometrika 75 (1988) 237-249], where at least a finite second moment is required. This excludes some important distributions, for example, those in the domain of attraction of a stable law with index between 1 and 2. In this article we use a method similar to Qin and Wong [Scand. J. Statist. 23 (1996) 209-219] to derive an empirical-likelihood-based confidence interval for the mean when the underlying distribution has heavy tails. Our method can easily be extended to obtain a confidence interval for any order o…
In this paper, we develop statistical inference techniques for the unknown coefficient functions and single-index parameters in single-index varying-coefficient models. We first estimate the nonparametric component via the local linear fitting, then construct an estimated empirical likelihood ratio function and hence obtain a maximum empirical likelihood estimator for the parametric component. Our estimator for parametric component is asymptotically efficient, and the estimator of nonparametric component has an optimal convergence rate. Our results provide ways to construct the confidence regi…
