This paper studies the high-dimensional mixed linear regression (MLR) where the output variable comes from one of the two linear regression models with an unknown mixing proportion and an unknown covariance structure of the random covariates. Building upon a high-dimensional EM algorithm, we propose an iterative procedure for estimating the two regression vectors and establish their rates of convergence. Based on the iterative estimators, we further construct debiased estimators and establish their asymptotic normality. For individual coordinates, confidence intervals centered at the debiased estimators are constructed.
Furthermore, a large-scale multiple testing procedure is proposed for testing the regression coefficients and is shown to control the false discovery rate (FDR) asymptotically. Simulation studies are carried out to examine the numerical performance of the proposed methods and their superiority over existing methods. The proposed methods are further illustrated through an analysis of a dataset of multiplex image cytometry, which investigates the interaction networks among the cellular phenotypes that include the expression levels of 20 epitopes or combinations of markers.
Null hypothesis significance tests and p values are widely used despite very strong arguments against their use in many contexts. Confidence intervals are often recommended as an alternative, but these do not achieve the objective of assessing the credibility of a hypothesis, and the distinction between confidence and probability is an unnecessary confusion. This paper proposes a more straightforward (probabilistic) definition of confidence, and suggests how the idea can be applied to whatever hypotheses are of interest to researchers. The relative merits of the different approaches are discus…
In many fields of research null hypothesis significance tests and p values are the accepted way of assessing the degree of certainty with which research results can be extrapolated beyond the sample studied. However, there are very serious concerns about the suitability of p values for this purpose. An alternative approach is to cite confidence intervals for a statistic of interest, but this does not directly tell readers how certain a hypothesis is. Here, I suggest how the framework used for confidence intervals could easily be extended to derive confidence levels, or "tentative probabilities…
This is the rejoinder to the discussion by Kennedy, Balakrishnan and Wasserman on the paper "On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning" published in Statistical Science.
Fisherian randomization inference is often dismissed as testing an uninteresting and implausible hypothesis: the sharp null of no effects whatsoever. We show that this view is overly narrow. Many randomization tests are also valid under a more general "bounded" null hypothesis under which all effects are weakly negative (or positive), thus accommodating heterogenous effects. By inverting such tests we can form one-sided confidence intervals for the maximum (or minimum) effect. These properties hold for all effect-increasing test statistics, which include both common statistics such as the mean…
We consider the problem of interval estimation of the odds ratio. An asymptotic confidence interval is widely applied in medical research. Unfortunately that confidence interval has a poor coverage probability: it is significantly smaller than the nominal confidence level. In this paper a new confidence interval is proposed. The construction needs only information on sample sizes and sample odds ratio. The coverage probability of the proposed confidence interval is at least the nominal confidence level.
We consider interval estimation of the difference between two binomial proportions. Several methods of constructing such an interval are known. Unfortunately those confidence intervals have poor coverage probability: it is significantly smaller than the nominal confidence level. In this paper a new confidence interval is proposed. The construction needs only information on sample sizes and sample difference between proportions. The coverage probability of the proposed confidence interval is at least the nominal confidence level. The new confidence interval is illustrated by a medical example.
Jędrzejczak et al. (2018) constructed a confidence interval for a ratio of quantiles coming from the Dagum distribution, which is frequently applied as a theoretical model in numerous income distribution analyses. The proposed interval is symmetric with respect to the ratio of sample quantiles, which result may be unsatisfactory in many practical applications. The search for a confidence interval with a smaller length led to the derivation of the shortest interval with the ends being asymmetric relative to the ratio of sample quantiles. In the paper, the existence of the shortest confidence in…
We evaluate the model averaged profile likelihood confidence intervals proposed by Fletcher and Turek (2011) in a simple situation in which there are two linear regression models over which we average. We obtain exact expressions for the coverage and the scaled expected length of the intervals and use these to compute these quantities in particular situations. We show that the Fletcher-Turek confidence intervals can have coverage well below the nominal coverage and expected length greater than that of the standard confidence interval with coverage equal to the same minimum coverage. In these s…
Out-of-bag error is commonly used as an estimate of generalisation error in ensemble-based learning models such as random forests. We present confidence intervals for this quantity using the delta-method-after-bootstrap and the jackknife-after-bootstrap techniques. These methods do not require growing any additional trees. We show that these new confidence intervals have improved coverage properties over the naive confidence interval, in real and simulated examples.
We propose simple formulas of confidence intervals for the Wald statistic, likelihood ratio statistic, and score statistic for a network meta-analysis. In addition, we consider resolutions for concerns that network meta-analyses with a small number of studies cannot hold a nominal confidence level. For a bias adjustment in analyses with a small number of studies, a Bartlett-type adjustment is a well-known method. Many Bartlett-type adjustment-type methods are based on maximum likelihood estimators. However, the network meta-analysis often uses the restricted maximum likelihood estimators that …
The current standard for confidence interval construction in the context of a possibly misspecified model is to use an interval based on the sandwich estimate of variance. These intervals provide asymptotically correct coverage, but small-sample coverage is known to be poor. By eliminating a plug-in assumption, we derive a pivot-based method for confidence interval construction under possibly misspecified models. When compared against confidence intervals generated by the sandwich estimate of variance, this method provides more accurate coverage of the pseudo-true parameter at small sample siz…
We introduce a new framework for creating point-wise confidence intervals for the distribution of event times for current status data. Existing methods are based on asymptotics. Our framework is based on binomial properties and motivates confidence intervals that are very simple to apply and are valid, i.e., guarantee nominal coverage. Although these confidence intervals are necessarily conservative for small sample sizes, asymptotically their coverage rate approaches the nominal one. This binomial framework also motivates approximately valid confidence intervals, and simulations show that the…
Contribution to the discussion of the paper "Causal inference using invariant prediction: identification and confidence intervals" by Peters, Bühlmann and Meinshausen, to appear in the Journal of the Royal Statistical Society, Series B.
Confidence interval procedures used in low dimensional settings are often inappropriate for high dimensional applications. When a large number of parameters are estimated, marginal confidence intervals associated with the most significant estimates have very low coverage rates: They are too small and centered at biased estimates. The problem of forming confidence intervals in high dimensional settings has previously been studied through the lens of selection adjustment. In this framework, the goal is to control the proportion of non-covering intervals formed for selected parameters. In this pa…
In economic research inequality measures based on ratios of quantiles are frequently applied to the analysis of income distributions. In the paper, we construct a confidence interval for such measures under the Dagum distribution which has been widely assumed as a~model for income distributions in empirical analyses Its properties are investigated on the basis of computer simulations. The constructed confidence interval is applied to the analysis of inequality income in Poland in 2015.
The factor estimation process is a really challenging task for non-linear models. Even whether researchers manage to successfully estimate model factors, they still must estimate their confidence intervals, which could require a high computational cost to turn them into informative measures. Some methods in the literature attempt to estimate regions within the estimation search space where factors may jointly exist and fit the real data (confidence contours), however, its estimation process raises several issues as the number of factors increases. Hence, in this paper, we focus on the estimati…
Several scientific fields including psychology are undergoing a replication crisis. There are many reasons for this problem, one of which is a misuse of p-values. There are several alternatives to p-values, and in this paper we describe a complement that is geared towards replication. In particular, we focus on confidence intervals for the probability that a parameter estimate will exceed a specified value in an exact replication study. These intervals convey uncertainty in a way that p-values and standard confidence intervals do not, and can help researchers to draw sounder scientific conclus…
When analyzing incomplete data, is it better to use multiple imputation (MI) or full information maximum likelihood (ML)? In large samples ML is clearly better, but in small samples ML's usefulness has been limited because ML commonly uses normal test statistics and confidence intervals that require large samples. We propose small-sample t-based ML confidence intervals that have good coverage and are shorter than t-based confidence intervals under MI. We also show that ML point estimates are less biased and more efficient than MI point estimates in small samples of bivariate normal data. With …
Hard thresholding, LASSO , adaptive LASSO and SCAD point estimators have been suggested for use in the linear regression context when most of the components of the regression parameter vector are believed to be zero, a sparsity type of assumption. Potscher and Schneider, 2010, Electronic Journal of Statistics, have considered the properties of fixed-width confidence intervals that include one of these point estimators (for all possible data values). They consider a normal linear regression model with orthogonal regressors and show that these confidence intervals are longer than the standard co…
Bootstrap smoothed (bagged) parameter estimators have been proposed as an improvement on estimators found after preliminary data-based model selection. The key result of Efron (2014) is a very convenient and widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. This approximation provides an easily computed guide to the accuracy of this estimator. In addition, Efron (2014) proposed a confidence interval centered on the bootstrap smoothed estimator, with width proportional to the estimate of this approximation to the standard d…
This short note argues that 95% confidence intervals for MCMC estimates can be obtained even without establishing a CLT, by multiplying their widths by 2.3.
We consider a general regression model, without a scale parameter. Our aim is to construct a confidence interval for a scalar parameter of interest $θ$ that utilizes the uncertain prior information that a distinct scalar parameter $τ$ takes the specified value $t$. This confidence interval should have good coverage properties. It should also have scaled expected length, where the scaling is with respect to the usual confidence interval, that (a) is substantially less than 1 when the prior information is correct, (b) has a maximum value that is not too large and (c) is close to 1 when the data …
We compare two ways of constructing confidence intervals for the moments-matching parameter estimates of a Gaussian spatio-temporal Ornstein-Uhlenbeck process. It was found that those obtained via pairwise likelihood approximations had lower coverages and were more prone to the curse of dimensionality as opposed to those from a parametric bootstrap procedure.
Consider panel data modelled by a linear random intercept model that includes a time-varying covariate. Suppose that we have uncertain prior information that this covariate is exogenous. We present a new confidence interval for the slope parameter that utilizes this uncertain prior information. This interval has minimum coverage probability very close to its nominal coverage. Let the scaled expected length of this new confidence interval be its expected length divided by the expected length of the confidence interval, with the same minimum coverage, constructed using the fixed effects model. T…
A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The confidence interval is consistent with an exact unconditional test that preserves the type-I error, and has improved power, particularly for smaller sample sizes, compared to the confidence interval by Chan & Zhang (1999). The improved performance of the proposed confidence interval is theoretically justified and demonstrated with simulations and examples. An R package is also distributed that implements the proposed methods along with other confidence interval estimators.
Penalized spline regression is a popular method for scatterplot smoothing, but there has long been a debate on how to construct confidence intervals for penalized spline fits. Due to the penalty, the fitted smooth curve is a biased estimate of the target function. Many methods, including Bayesian intervals and the simple-shift bias-reduction, have been proposed to upgrade the coverage of the confidence intervals, but these methods usually fail to adequately improve the situation at predictor values where the function is sharply curved. In this paper, we develop a novel approach to improving th…
Introductory texts on statistics typically only cover the classical "two sigma" confidence interval for the mean value and do not describe methods to obtain confidence intervals for other estimators. The present technical report fills this gap by first defining different methods for the construction of confidence intervals, and then by their application to a binomial proportion, the mean value, and to arbitrary estimators. Beside the frequentist approach, the likelihood ratio and the highest posterior density approach are explained. Two methods to estimate the variance of general maximum likel…
Given $m$ unknown parameters with corresponding independent estimators, the Benjamini-Hochberg (BH) procedure can be used to classify the sign of parameters such that the expected proportion of erroneous directional decisions (directional FDR) is controlled at a preset level $q$. More ambitiously, our goal is to construct sign-determining confidence intervals---instead of only classifying the sign---such that the expected proportion of non-covering constructed intervals (FCR) is controlled. We suggest a valid procedure which adjusts a marginal confidence interval in order to construct a maximu…
This study aims to evaluate the performance of power in the likelihood ratio test for changepoint detection by bootstrap sampling, and proposes a hypothesis test based on bootstrapped confidence interval lengths. Assuming i.i.d normally distributed errors, and using the bootstrap method, the changepoint sampling distribution is estimated. Furthermore, this study describes a method to estimate a data set with no changepoint to form the null sampling distribution. With the null sampling distribution, and the distribution of the estimated changepoint, critical values and power calculations can be…
The goal of any estimation study is an interval estimation of a the parameter(s) of interest. These estimations are mostly expressed using empirical confidence intervals that are based on sample point estimates of the corresponding parameter(s). In contrast, calculations of the necessary sample size usually use expected confidence intervals that are based on the expected value of the parameter(s). The approach that guarantees the required probability of the required width of empirical confidence interval is known at least since 1989. However, till now, this approach is not implemented for most…
We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.
In the analysis of clustered and longitudinal data, which includes a covariate that varies both between and within clusters (e.g. time-varying covariate in longitudinal data), a Hausman pretest is commonly used to decide whether subsequent inference is made using the linear random intercept model or the fixed effects model. We assess the effect of this pretest on the coverage probability and expected length of a confidence interval for the slope parameter. Our results show that for the small levels of significance of the Hausman pretest commonly used in applications, the minimum coverage proba…
This report compares two tests of second-order stationarity through simulation. It also provides several examples of localised autocovariances and their approximate confidence intervals on different real and simulated data sets. An empirical verification of an asymptotic Gaussianity result is also demonstrated. The commands use to produce figures in a companion paper are also described.
We congratulate the authors on their exciting paper, which introduces a novel idea for assessing the estimation bias in causal estimates. Doubly robust estimators are now part of the standard set of tools in causal inference, but a typical analysis stops with an estimate and a confidence interval. The authors give an approach for a unique type of model-checking that allows the user to check whether the bias is sufficiently small with respect to the standard error, which is generally required for confidence intervals to be reliable.
Practical or scientific considerations often lead to selecting a subset of parameters as ``important.'' Inferences about those parameters often are based on the same data used to select them in the first place. That can make the reported uncertainties deceptively optimistic: confidence intervals that ignore selection generally have less than their nominal coverage probability. Controlling the probability that one or more intervals for selected parameters do not cover---the ``simultaneous over the selected'' (SoS) error rate---is crucial in many scientific problems. Intervals that control the S…
Split sample methods have recently been put forward as a way to reduce the coverage oscillations that haunt confidence intervals for parameters of lattice distributions, such as the binomial and Poisson distributions. We study split sample intervals in the binomial setting, showing that these intervals can be viewed as being based on adding discrete random noise to the data. It is shown that they can be improved upon by using noise with a continuous distribution instead, regardless of whether the randomization is determined by the data or an external source of randomness. We compare split samp…
The age of big data has produced data sets that are computationally expensive to analyze and store. Algorithmic leveraging proposes that we sample observations from the original data set to generate a representative data set and then perform analysis on the representative data set. In this paper, we present efficient algorithms for constructing finite sample confidence intervals for each algorithmic leveraging estimated regression coefficient, with asymptotic coverage guarantees. In simulations, we confirm empirically that the confidence intervals have the desired coverage probabilities, while…
Interval estimation of the probability of success in a Binomial model is considered. Zieliński (2018) showed that the confidence interval which uses information about non-homogeneity of the sample is better than the classical one. In the following paper the shortest confidence interval for non-homogenous sample is constructed.
It is in general challenging to provide confidence intervals for individual variables in high-dimensional regression without making strict or unverifiable assumptions on the design matrix. We show here that a "group-bound" confidence interval can be derived without making any assumptions on the design matrix. The lower bound for the regression coefficient of individual variables can be derived via linear programming. The idea also generalises naturally to groups of variables, where we can derive a one-sided confidence interval for the joint effect of a group. While the confidence intervals of …
Consider a linear regression model with n-dimensional response vector, regression parameter β= (β_1, ..., β_p) and independent and identically N(0, σ^2) distributed errors. Suppose that the parameter of interest is θ= a^T βwhere a is a specified vector. Define the parameter τ= c^T β- t where c and t are specified. Also suppose that we have uncertain prior information that τ= 0. Part of our evaluation of a frequentist confidence interval for θis the ratio (expected length of this confidence interval)/(expected length of standard 1-αconfidence interval), which we call the scaled expected length …
Confidence interval (CI) methods for stratified bilateral studies use intraclass correlation to avoid misleading results. In this article, we propose four CI methods (sample-size weighted global MLE-based Wald-type CI, complete MLE-based Wald-type CI, profile likelihood CI, and complete MLE-based score CI) to investigate CIs of proportion ratios to clinical trial design with stratified bilateral data under Dallal's intraclass model. Monte Carlo simulations are performed, and the complete MLE-based score confidence interval (CS) method yields a robust outcome. Lastly, a real data example is con…
Confidence intervals are a popular way to visualize and analyze data distributions. Unlike p-values, they can convey information both about statistical significance as well as effect size. However, very little work exists on applying confidence intervals to multivariate data. In this paper we define confidence intervals for multivariate data that extend the one-dimensional definition in a natural way. In our definition every variable is associated with its own confidence interval as usual, but a data vector can be outside of a few of these, and still be considered to be within the confidence a…
We compare the following two sources of poor coverage of post-model-selection confidence intervals: the preliminary data-based model selection sometimes chooses the wrong model and the data used to choose the model is re-used for the construction of the confidence interval.
Linear combinations of multinomial probabilities, such as those resulting from contingency tables, are of use when evaluating classification system performance. While large sample inference methods for these combinations exist, small sample methods exist only for regions on the multinomial parameter space instead of the linear combinations. However, in medical classification problems it is common to have small samples necessitating a small sample confidence interval on linear combinations of multinomial probabilities. Therefore, in this paper we derive an exact confidence interval, through the…
A new method is proposed for the correction of confidence intervals when the original interval does not have the correct nominal coverage probabilities in the frequentist sense. The proposed method is general and does not require any distributional assumptions. It can be applied to both frequentist and Bayesian inference where interval estimates are desired. We provide theoretical results for the consistency of the proposed estimator, and give two complex examples, on confidence interval correction for composite likelihood estimators and in approximate Bayesian computation (ABC), to demonstrat…
The purpose of this paper is to propose methodologies for statistical inference of low-dimensional parameters with high-dimensional data. We focus on constructing confidence intervals for individual coefficients and linear combinations of several of them in a linear regression model, although our ideas are applicable in a much broad context. The theoretical results presented here provide sufficient conditions for the asymptotic normality of the proposed estimators along with a consistent estimator for their finite-dimensional covariance matrices. These sufficient conditions allow the number of…
In this short note I apply the methodology of game-theoretic probability to calculating non-asymptotic confidence intervals for the coefficient of a simple first order scalar autoregressive model. The most distinctive feature of the proposed procedure is that with high probability it produces confidence intervals that always cover the true parameter value when applied sequentially.
University of Alberta（阿尔伯塔大学）；DeepMind（深度思维）；Google Research（谷歌研究院）
We study high-confidence behavior-agnostic off-policy evaluation in reinforcement learning, where the goal is to estimate a confidence interval on a target policy's value, given only access to a static experience dataset collected by unknown behavior policies. Starting from a function space embedding of the linear program formulation of the $Q$-function, we obtain an optimization problem with generalized estimating equation constraints. By applying the generalized empirical likelihood method to the resulting Lagrangian, we propose CoinDICE, a novel and efficient algorithm for computing confide…
