Empirical-likelihood-based confidence intervals for a mean were introduced by Owen [Biometrika 75 (1988) 237-249], where at least a finite second moment is required. This excludes some important distributions, for example, those in the domain of attraction of a stable law with index between 1 and 2. In this article we use a method similar to Qin and Wong [Scand.
J. Statist. 23 (1996) 209-219] to derive an empirical-likelihood-based confidence interval for the mean when the underlying distribution has heavy tails. Our method can easily be extended to obtain a confidence interval for any order of moment of a heavy-tailed distribution.
Let $X_1, \ldots, X_n$ be some i.i.d. observations from a heavy tailed distribution $F$, i.e. such that the common distribution of the excesses over a high threshold $u_n$ can be approximated by a Generalized Pareto Distribution $G_{γ,σ_n}$ with $γ>0$. This work is devoted to the problem of finding confidence regions for the couple $(γ,σ_n)$ : combining the empirical likelihood methodology with estimation equations (close but not identical to the likelihood equations) introduced by J. Zhang (Australian and New Zealand J. Stat n.49(1), 2007), asymptotically valid confidence regions for $(γ,σ_n)…
A variety of estimators for the parameters of the Generalized Pareto distribution, the approximating distribution for excesses over a high threshold, have been proposed, always assuming the underlying data to be independent. We recently proved that the likelihood moment estimators are consistent estimators for the parameters of the Generalized Pareto distribution for the case where the underlying data arises from a (stationary) linear process with heavy-tailed innovations. In this paper we derive the bivariate asymptotic normality under some additional assumptions and give an explicit example …
We present new estimators of the mean of a real valued random variable, based on PAC-Bayesian iterative truncation. We analyze the non-asymptotic minimax properties of the deviations of estimators for distributions having either a bounded variance or a bounded kurtosis. It turns out that these minimax deviations are of the same order as the deviations of the empirical mean estimator of a Gaussian distribution. Nevertheless, the empirical mean itself performs poorly at high confidence levels for the worst distribution with a given variance or kurtosis (which turns out to be heavy tailed). To ob…
By the modified directed likelihood, higher order accurate confidence limits for a scalar parameter are obtained from the likelihood. They are conveniently described in terms of a confidence distribution, that is a sample dependent distribution function on the parameter space. In this paper we explore a different route to accurate confidence limits via tail-symmetric confidence curves, that is curves that describe equal tailed intervals at any level. Instead of modifying the directed likelihood, we consider inversion of the log-likelihood ratio when evaluated at the median of the maximum likel…
This paper considers Bayesian multiple testing under sparsity for polynomial-tailed distributions satisfying a monotone likelihood ratio property. Included in this class of distributions are the Student's t, the Pareto, and many other distributions. We prove some general asymptotic optimality results under fixed and random thresholding. As examples of these general results, we establish the Bayesian asymptotic optimality of several multiple testing procedures in the literature for appropriately chosen false discovery rate levels. We also show by simulation that the Benjamini-Hochberg procedure…
We propose a sparse coefficient estimation and automated model selection procedure for autoregressive (AR) processes with heavy-tailed innovations based on penalized conditional maximum likelihood. Under mild moment conditions on the innovation processes, the penalized conditional maximum likelihood estimator (PCMLE) satisfies a strong consistency, $O_P(N^{-1/2})$ consistency, and the oracle properties, where N is the sample size. We have the freedom in choosing penalty functions based on the weak conditions on them. Two penalty functions, least absolute shrinkage and selection operator (LASSO…
The widely claimed replicability crisis in science may lead to revised standards of significance. The customary frequentist confidence intervals, calibrated through hypothetical repetitions of the experiment that is supposed to have produced the data at hand, rely on a feeble concept of replicability. In particular, contradictory conclusions may be reached when a substantial enlargement of the study is undertaken. To redefine statistical confidence in such a way that inferential conclusions are non-contradictory, with large enough probability, under enlargements of the sample, we give a new re…
Empirical likelihood is a well-known nonparametric method in statistics and has been widely applied in statistical inference. The method has been employed by Lu and Peng (2002) to constructing confidence intervals for the tail index of a heavy-tailed distribution. It is demonstrated in Lu and Peng (2002) that the empirical likelihood-based confidence intervals performs better than confidence intervals based on normal approximation in terms of the coverage probability. In general, the empirical likelihood method can be hindered by its imprecision in the coverage probability when the sample size…
We use a logical device called the Dutch Book to establish epistemic confidence, defined as the sense of confidence \emph{in an observed} confidence interval. This epistemic property is unavailable -- or even denied -- in orthodox frequentist inference. In financial markets, including the betting market, the Dutch Book is also known as arbitrage or risk-free profitable transaction. A numerical confidence is deemed epistemic if its use as a betting price is protected from the Dutch Book by an external agent. Theoretically, to construct the Dutch Book, the agent must exploit unused information a…
The construction of the simultaneous confidence bands for the integrated hazard function is considered. The Nelson--Aalen estimator is used. The simultaneous confidence bands based on bootstrap methods are presented. Two methods of construction of such confidence bands are proposed. The weird bootstrap method is used for resampling. Simulations are made to compare the actual coverage probability of the bootstrap and the asymptotic simultaneous confidence bands. It is shown that the equal--tailed bootstrap confidence band has the coverage probability closest to the nominal one. We also present …
Thus far, likelihood-based interval estimates for quantiles have not been studied in the literature on interval censored case 2 data and partly interval censored data, and, in this context, the use of smoothing has not been considered for any type of censored data. This article constructs smoothed weighted empirical likelihood ratio confidence intervals (WELRCI) for quantiles in a unified framework for various types of censored data, including right censored data, doubly censored data, interval censored data and partly interval censored data. The fourth order expansion of the weighted empirica…
We consider a likelihood ratio method for testing whether a monotone baseline hazard function in the Cox model has a particular value at a fixed point. The characterization of the estimators involved is provided both in the nondecreasing and the nonincreasing setting. These characterizations facilitate the derivation of the asymptotic distribution of the likelihood ratio test, which is identical in the nondecreasing and in the nonincreasing case. The asymptotic distribution of the likelihood ratio test enables, via inversion, the construction of pointwise confidence intervals. Simulations show…
Empirical likelihood approach is one of non-parametric statistical methods, which is applied to the hypothesis testing or construction of confidence regions for pivotal unknown quantities. This method has been applied to the case of independent identically distributed random variables and second order stationary processes. In recent years, we observe heavy-tailed data in many fields. To model such data suitably, we consider symmetric scalar and multivariate $α$-stable linear processes generated by infinite variance innovation sequence. We use a Whittle likelihood type estimating function in th…
In this paper we consider the behavior of Kalman Filter state estimates in the case of distribution with heavy tails .The simulated linear state space models with Gaussian measurement noises were used. Gaussian noises in state equation are replaced by components with alpha-stable distribution with di erent parameters alpha and beta. We consider the case when "all parameters are known" and two methods of parameters estimation are compared: the maximum likelihood estimator (MLE) and the expectation- maximization algorithm (EM). It was shown that in cases of large deviation from Gaussian distribu…
We use bias-reduced estimators of high quantiles, of heavy-tailed distributions, to introduce a new estimator of the mean in the case of infinite second moment. The asymptotic normality of the proposed estimator is established and checked, in a simulation study, by four of the most popular goodness-of-fit tests for different sample sizes. Moreover, we compare, in terms of bias and mean squared error, our estimator with Peng's estimator (Peng, 2001) and we evaluate the accuracy of some resulting confidence intervals.
A novel statistical method is proposed and investigated for estimating a heavy tailed density under mild smoothness assumptions. Statistical analyses of heavy-tailed distributions are susceptible to the problem of sparse information in the tail of the distribution getting washed away by unrelated features of a hefty bulk. The proposed Bayesian method avoids this problem by incorporating smoothness and tail regularization through a carefully specified semiparametric prior distribution, and is able to consistently estimate both the density function and its tail index at near minimax optimal rate…
Introduction: there is an ongoing debate about directional inference of two-sided hypothesis tests for which some authors argue that rejecting $θ= θ_0$ does not allow to conclude that $θ> θ_0$ or $θ< θ_0$ but only that $θ\neq θ_0$, while others argue that this is a minor error without practical consequence. Discussion: new elements are brought to the debate. It is shown that the directional interpretation of some non-directional hypothesis tests about Receiver Operating Characteristic (ROC) and survival curves may lead to inflated type III error rates with a probability of concluding that a di…
The block maxima method in extreme value theory consists of fitting an extreme value distribution to a sample of block maxima extracted from a time series. Traditionally, the maxima are taken over disjoint blocks of observations. Alternatively, the blocks can be chosen to slide through the observation period, yielding a larger number of overlapping blocks. Inference based on sliding blocks is found to be more efficient than inference based on disjoint blocks. The asymptotic variance of the maximum likelihood estimator of the Fréchet shape parameter is reduced by more than 18%. Interestingly, t…
In this work, we focus on a variant of the generalized linear model (GLM) called corrupted GLM (CGLM) with heavy-tailed features and responses. To robustify the statistical inference on this model, we propose to apply $\ell_4$-norm shrinkage to the feature vectors in the low-dimensional regime and apply elementwise shrinkage to them in the high-dimensional regime. Under bounded fourth moment assumptions, we show that the maximum likelihood estimator (MLE) based on the shrunk data enjoys nearly the minimax optimal rate with an exponential deviation bound. Our simulations demonstrate that the pr…
Real-world networks are generally claimed to be scale-free, meaning that the degree distributions follow the classical power-law, at least asymptotically. Yet, closer observation shows that the classical power-law distribution is often inadequate to meet the data characteristics due to the existence of a clearly identifiable non-linearity in the entire degree distribution in the log-log scale. The present paper proposes a new variant of the popular heavy-tailed Lomax distribution which we named as the Modified Lomax (MLM) distribution that can efficiently capture the crucial aspect of heavy-ta…
The traditional activity of model selection aims at discovering a single model superior to other candidate models. In the presence of pronounced noise, however, multiple models are often found to explain the same data equally well. To resolve this model selection ambiguity, we introduce the general approach of model selection confidence sets (MSCSs) based on likelihood ratio testing. A MSCS is defined as a list of models statistically indistinguishable from the true model at a user-specified level of confidence, which extends the familiar notion of confidence intervals to the model-selection f…
Maximum likelihood estimates and corresponding confidence regions of the estimates are commonly used in statistical inference. In practice, people often construct approximate confidence regions with the Fisher information at given sample data based on the asymptotic normal distribution of the MLE (maximum likelihood estimate). Two common Fisher information matrices (FIMs, for multivariate parameters) are the observed FIM (the Hessian matrix of negative log-likelihood function) and the expected FIM (the expectation of the observed FIM). In this article, we prove that under certain conditions an…
We study nonparametric isotonic confidence intervals for monotone functions. In Banerjee and Wellner (2001) pointwise confidence intervals, based on likelihood ratio tests for the restricted and unrestricted MLE in the current status model, are introduced. We extend the method to the treatment of other models with monotone functions, and demonstrate our method by a new proof of the results in Banerjee and Wellner (2001) and also by constructing confidence intervals for monotone densities, for which still theory had to be developed. For the latter model we prove that the limit distribution of t…
When in a full exponential family the maximum likelihood estimate (MLE) does not exist, the MLE may exist in the Barndorff-Nielsen completion of the family. We propose a practical algorithm for finding the MLE in the completion based on repeated linear programming using the R contributed package rcdd and illustrate it with two generalized linear model examples. When the MLE for the null hypothesis lies in the completion, likelihood ratio tests of model comparison are almost unchanged from the usual case. Only the degrees of freedom need to be adjusted. When the MLE lies in the completion, conf…
Fisher's Method of Maximum Likelihood is shown to be a procedure for the construction of likelihood intervals or regions, instead of a procedure of point estimation. Based on Fisher's articles and books it is justified that by estimation Fisher meant the construction of likelihood intervals or regions from appropriate likelihood function and that an estimate is a statistic, that is, a function from a sample space to a parameter space such that the likelihood function obtained from the sampling distribution of the statistic at the observed value of the statistic is used to construct likelihood …
This paper investigates the theoretical underpinnings of two fundamental statistical inference problems, the construction of confidence sets and large-scale simultaneous hypothesis testing, in the presence of heavy-tailed data. With heavy-tailed observation noise, finite sample properties of the least squares-based methods, typified by the sample mean, are suboptimal both theoretically and empirically. In this paper, we demonstrate that the adaptive Huber regression, integrated with the multiplier bootstrap procedure, provides a useful robust alternative to the method of least squares. Our the…
In this paper, we develop statistical inference techniques for the unknown coefficient functions and single-index parameters in single-index varying-coefficient models. We first estimate the nonparametric component via the local linear fitting, then construct an estimated empirical likelihood ratio function and hence obtain a maximum empirical likelihood estimator for the parametric component. Our estimator for parametric component is asymptotically efficient, and the estimator of nonparametric component has an optimal convergence rate. Our results provide ways to construct the confidence regi…
We discuss a new way of constructing pointwise confidence intervals for the distribution function in the current status model. The confidence intervals are based on the smoothed maximum likelihood estimator (SMLE) and constructed using bootstrap methods. Other methods to construct confidence intervals, using the non-standard limit distribution of the (restricted) MLE, are compared to our approach via simulations and real data applications.
Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the ma…
We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.
In this paper, we propose robust stochastic algorithms for solving convex compositional problems of the form $f(\E_ξg(\cdot; ξ)) + r(\cdot)$ by establishing {\bf sub-Gaussian confidence bounds} under weak assumptions about the tails of noise distribution, i.e., {\bf heavy-tailed noise} with bounded second-order moments. One can achieve this goal by using an existing boosting strategy that boosts a low probability convergence result into a high probability result. However, piecing together existing results for solving compositional problems suffers from several drawbacks: (i) the boosting techn…
We propose simple formulas of confidence intervals for the Wald statistic, likelihood ratio statistic, and score statistic for a network meta-analysis. In addition, we consider resolutions for concerns that network meta-analyses with a small number of studies cannot hold a nominal confidence level. For a bias adjustment in analyses with a small number of studies, a Bartlett-type adjustment is a well-known method. Many Bartlett-type adjustment-type methods are based on maximum likelihood estimators. However, the network meta-analysis often uses the restricted maximum likelihood estimators that …
Profile likelihood confidence intervals are a robust alternative to Wald's method if the asymptotic properties of the maximum likelihood estimator are not met. However, the constrained optimization problem defining profile likelihood confidence intervals can be difficult to solve in these situations, because the likelihood function may exhibit unfavorable properties. As a result, existing methods may be inefficient and yield misleading results. In this paper, we address this problem by computing profile likelihood confidence intervals via a trust-region approach, where steps computed based on …
Estimating high quantiles plays an important role in the context of risk management. This involves extrapolation of an unknown distribution function. In this paper we propose three methods, namely, the normal approximation method, the likelihood ratio method and the data tilting method, to construct confidence regions for high quantiles of a heavy tailed distribution. A simulation study prefers the data tilting method.
This study aims to evaluate the performance of power in the likelihood ratio test for changepoint detection by bootstrap sampling, and proposes a hypothesis test based on bootstrapped confidence interval lengths. Assuming i.i.d normally distributed errors, and using the bootstrap method, the changepoint sampling distribution is estimated. Furthermore, this study describes a method to estimate a data set with no changepoint to form the null sampling distribution. With the null sampling distribution, and the distribution of the estimated changepoint, critical values and power calculations can be…
We evaluate the model averaged profile likelihood confidence intervals proposed by Fletcher and Turek (2011) in a simple situation in which there are two linear regression models over which we average. We obtain exact expressions for the coverage and the scaled expected length of the intervals and use these to compute these quantities in particular situations. We show that the Fletcher-Turek confidence intervals can have coverage well below the nominal coverage and expected length greater than that of the standard confidence interval with coverage equal to the same minimum coverage. In these s…
The asymptotic results pertaining to the distribution of the log likelihood ratio allow for the creation of a confidence region, which is a general extension of the confidence interval. Two and three dimensional regions can be displayed visually in order to describe the plausible region of the parameters of interest simultaneously. While most advanced statistical textbooks on inference discuss these asymptotic confidence regions, there is no exploration of how to numerically compute these regions for graphical purposes. This article demonstrates the application of a simple trigonometric identi…
The role played by the composite analogue of the log likelihood ratio in hypothesis testing and in setting confidence regions is not as prominent as it is in the canonical likelihood setting, since its asymptotic distribution depends on the unknown parameter. Approximate pivots based on the composite log likelihood ratio can be derived by using asymptotic arguments. However, the actual distribution of such pivots may differ considerably from the asymptotic reference, leading to tests and confidence regions whose levels are distant from the nominal ones. The use of bootstrap rather than asympto…
We present a new approach for inference about a log-concave distribution: Instead of using the method of maximum likelihood, we propose to incorporate the log-concavity constraint in an appropriate nonparametric confidence set for the cdf $F$. This approach has the advantage that it automatically provides a measure of statistical uncertainty and it thus overcomes a marked limitation of the maximum likelihood estimate. In particular, we show how to construct confidence bands for the density that have a finite sample guaranteed confidence level. The nonparametric confidence set for $F$ which we …
The subject of tail estimation for randomly censored data from a heavy tailed distribution receives growing attention, motivated by applications for instance in actuarial statistics. The bias of the available estimators of the extreme value index can be substantial and depends strongly on the amount of censoring. We review the available estimators, propose a new bias reduced estimator, and show how shrinkage estimation can help to keep the MSE under control. A bootstrap algorithm is proposed to construct confidence intervals. We compare these new proposals with the existing estimators through …
When analyzing incomplete data, is it better to use multiple imputation (MI) or full information maximum likelihood (ML)? In large samples ML is clearly better, but in small samples ML's usefulness has been limited because ML commonly uses normal test statistics and confidence intervals that require large samples. We propose small-sample t-based ML confidence intervals that have good coverage and are shorter than t-based confidence intervals under MI. We also show that ML point estimates are less biased and more efficient than MI point estimates in small samples of bivariate normal data. With …
This article studies local and global inference for smoothing spline estimation in a unified asymptotic framework. We first introduce a new technical tool called functional Bahadur representation, which significantly generalizes the traditional Bahadur representation in parametric models, that is, Bahadur [Ann. Inst. Statist. Math. 37 (1966) 577-580]. Equipped with this tool, we develop four interconnected procedures for inference: (i) pointwise confidence interval; (ii) local likelihood ratio testing; (iii) simultaneous confidence band; (iv) global likelihood ratio testing. In particular, our…
Inter-event times of various human behavior are apparently non-Poissonian and obey long-tailed distributions as opposed to exponential distributions, which correspond to Poisson processes. It has been suggested that human individuals may switch between different states in each of which they are regarded to generate events obeying a Poisson process. If this is the case, inter-event times should approximately obey a mixture of exponential distributions with different parameter values. In the present study, we introduce the minimum description length principle to compare mixtures of exponential d…
