This paper describes an R package implementing large sample tests and confidence intervals (based on the central limit theorem) for various parameters. The one and two sample mean and variance contexts are considered. The statistics for all the tests are expressed in the same form, which facilitates their presentation. In the variance parameter cases, the asymptotic robustness of the classical tests depends on the departure of the data distribution from normality measured in terms of the kurtosis of the distribution.
This paper presents the R package MCS which implements the Model Confidence Set (MCS) procedure recently developed by Hansen et al. (2011). The Hansen's procedure consists on a sequence of tests which permits to construct a set of 'superior' models, where the null hypothesis of Equal Predictive Ability (EPA) is not rejected at a certain confidence level. The EPA statistic tests is calculated for an arbitrary loss function, meaning that we could test models on various aspects, for example punctual forecasts. The relevance of the package is shown using an example which aims at illustrating in de…
We develop an R package SPQR that implements the semi-parametric quantile regression (SPQR) method in Xu and Reich (2021). The method begins by fitting a flexible density regression model using monotonic splines whose weights are modeled as data-dependent functions using artificial neural networks. Subsequently, estimates of conditional density and quantile process can all be obtained. Unlike many approaches to quantile regression that assume a linear model, SPQR allows for virtually any relationship between the covariates and the response distribution including non-linear effects and differen…
Poznan University of Technology（波兹南理工大学）；Polish Academy of Sciences（波兰科学院）；MNM Bioscience（MNM生物科学公司）
Machine learning applications, especially in the fields of me\-di\-cine and social sciences, are slowly being subjected to increasing scrutiny. Similarly to sample size planning performed in clinical and social studies, lawmakers and funding agencies may expect statistical uncertainty estimations in machine learning applications that impact society. In this paper, we present an easy-to-use python package and web application for estimating prediction confidence intervals. The package offers eight different procedures to determine and justify the sample size and confidence of predictions from ho…
A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The confidence interval is consistent with an exact unconditional test that preserves the type-I error, and has improved power, particularly for smaller sample sizes, compared to the confidence interval by Chan & Zhang (1999). The improved performance of the proposed confidence interval is theoretically justified and demonstrated with simulations and examples. An R package is also distributed that implements the proposed methods along with other confidence interval estimators.
In learn-as-you-go (LAGO) adaptive studies, the intervention is a complex package consisting of multiple components, and is adapted in stages during the study based on past outcome data. This design formalizes standard practice, and desires for practice, in public health intervention studies. An effective intervention package is sought, while minimizing intervention package cost. When analyzing data from a learn-as-you-go study, the interventions in later stages depend upon the outcomes in the previous stages, violating standard statistical theory. We develop methods for estimating the interve…
The asymptotic behaviour of the commonly used bootstrap percentile confidence interval is investigated when the parameters are subject to linear inequality constraints. We concentrate on the important one- and two-sample problems with data generated from general parametric distributions in the natural exponential family. The focus of this paper is on quantifying the coverage probabilities of the parametric bootstrap percentile confidence intervals, in particular their limiting behaviour near boundaries. We propose a local asymptotic framework to study this subtle coverage behaviour. Under this…
The multivariate Fay-Herriot model is quite effective in combining information through correlations among small area survey estimates of related variables or historical survey estimates of the same variable or both. Though the literature on small area estimation is already very rich, construction of second-order efficient confidence intervals from multivariate models have so far received very little attention. In this paper, we develop a parametric bootstrap method for constructing a second-order efficient confidence interval for a general linear combination of small area means using the multi…
The goal of this paper is to develop a practical and general-purpose approach to construct confidence intervals for differentially private parametric estimation. We find that the parametric bootstrap is a simple and effective solution. It cleanly reasons about variability of both the data sample and the randomized privacy mechanism and applies "out of the box" to a wide class of private estimation routines. It can also help correct bias caused by clipping data to limit sensitivity. We prove that the parametric bootstrap gives consistent confidence intervals in two broadly relevant settings, in…
This paper presents two results concerning uniform confidence intervals for the tail index and the extreme quantile. First, we show that it is impossible to construct a length-optimal confidence interval satisfying the correct uniform coverage over a local non-parametric family of tail distributions. Second, in light of the impossibility result, we construct honest confidence intervals that are uniformly valid by incorporating the worst-case bias in the local non-parametric family. The proposed method is applied to simulated data and a real data set of National Vital Statistics from National C…
We present an efficient method of calculating exact confidence intervals for the hypergeometric parameter representing the number of "successes," or "special items," in the population. The method inverts minimum-width acceptance intervals after shifting them to make their endpoints nondecreasing while preserving their level. The resulting set of confidence intervals achieves minimum possible average size, and even in comparison with confidence sets not required to be intervals it attains the minimum possible cardinality most of the time, and always within $1$. The method compares favorably wit…
We present the calibrated-projection MATLAB package implementing the method to construct confidence intervals proposed by Kaido, Molinari and Stoye (2017). This manual provides details on how to use the package for inference on projections of partially identified parameters. It also explains how to use the MATLAB functions we developed to compute confidence intervals on solutions of nonlinear optimization problems with estimated constraints.
ruptures is a Python library for offline change point detection. This package provides methods for the analysis and segmentation of non-stationary signals. Implemented algorithms include exact and approximate detection for various parametric and non-parametric models. ruptures focuses on ease of use by providing a well-documented and consistent interface. In addition, thanks to its modular structure, different algorithms and models can be connected and extended within this package.
This paper aims to construct a valid and efficient confidence interval for the extrema of parameters under privacy protection. The usual statistical inference on the extrema of parameters often suffers from the selection bias issue, and the problem becomes more acute, as in many application scenarios of extrema parameters, we often need to protect the privacy of the data. In this paper, we focus on the exponential family of distributions and propose a privatized parametric bootstrap method to address selection bias in the extrema of parameters problem under the scheme of differential privacy. …
Using the mathematical framework of information geometry, we introduce a novel method which allows one to efficiently determine the exact shape of simultaneous confidence regions for non-linearly parametrised models. Furthermore, we show how pointwise confidence bands around the model predictions can be constructed from detailed knowledge of the exact confidence region with little additional computational effort. We exemplify our methods using inference problems in cosmology and epidemic modelling. An open source implementation of the developed schemes is publicly available via the Information…
We investigate an application of the Tukey's methodology in Theil's regression to obtain a confidence interval for the true slope in the straight line regression model with not necessarily normal errors. This specific approach is implemented since 2005 in a package of the software R; however, without any theoretical background. We illustrate by Monte Carlo simulations, that this methodology, unlike the classical Theil's approach based on Kendall's tau, seriously deflates the true confidence level of the resulting interval. We provide also rigorous proofs in case of four data points (in general…
The R package quantreg.nonpar implements nonparametric quantile regression methods to estimate and make inference on partially linear quantile models. quantreg.nonpar obtains point estimates of the conditional quantile function and its derivatives based on series approximations to the nonparametric part of the model. It also provides pointwise and uniform confidence intervals over a region of covariate values and/or quantile indices for the same functions using analytical and resampling methods. This paper serves as an introduction to the package and displays basic functionality of the functio…
We introduce a new adjusted residual maximum likelihood method (REML) in the context of producing an empirical Bayes (EB) confidence interval for a normal mean, a problem of great interest in different small area applications. Like other rival empirical Bayes confidence intervals such as the well-known parametric bootstrap empirical Bayes method, the proposed interval is second-order correct, that is, the proposed interval has a coverage error of order $O(m^{-{3}/{2}})$. Moreover, the proposed interval is carefully constructed so that it always produces an interval shorter than the correspondi…
A confidence distribution is a complete tool for making frequentist inference for a parameter of interest $ψ$ based on an assumed parametric model. Indeed, it allows to reach point estimates, to assess their precision, to set up tests along with measures of evidence for statements of the type "$ψ> ψ_0$" or "$ψ_1 \leq ψ\leq ψ_2$", to derive confidence intervals, comparing the parameter of interest with other parameters from other studies, etc. The aim of this contribution is to discuss robust confidence distributions derived from unbiased $M-$estimating functions, which provide robust inference…
Institute for Employment Research（就业研究所）；University of Maryland（马里兰大学）；University of Pennsylvania（宾夕法尼亚大学）；Apple（苹果公司）；Harvard John A. Paulson School of Engineering and Applied Sciences（哈佛大学约翰·A·保尔森工程与应用科学学院）；Boston University（波士顿大学）
Differential privacy is a restriction on data processing algorithms that provides strong confidentiality guarantees for individual records in the data. However, research on proper statistical inference, that is, research on properly quantifying the uncertainty of the (noisy) sample estimate regarding the true value in the population, is currently still limited. This paper proposes and evaluates several strategies to compute valid differentially private confidence intervals for the median. Instead of computing a differentially private point estimate and deriving its uncertainty, we directly est…
Frequentist prediction intervals for random-effects meta-analysis via confidence-distribution propagation
The Institute of Statistical Mathematics（日本统计科学研究所）；The Graduate University for Advanced Studies（高级研究院）；Institute of Medical Biometry and Statistics, Medical Faculty and Medical Center – University of Freiburg（弗赖堡大学医学院医学中心医学生物统计学与统计研究所）
The R package frailtySurv for simulating and fitting semi-parametric shared frailty models is introduced. Package frailtySurv implements semi-parametric consistent estimators for a variety of frailty distributions, including gamma, log-normal, inverse Gaussian and power variance function, and provides consistent estimators of the standard errors of the parameters' estimators. The parameters' estimators are asymptotically normally distributed, and therefore statistical inference based on the results of this package, such as hypothesis testing and confidence intervals, can be performed using the…
Estimation of linear functionals from observed data is an important task in many subjects. Juditsky & Nemirovski [The Annals of Statistics 37.5A (2009): 2278-2300] propose a framework for non-parametric estimation of linear functionals in a very general setting, with nearly minimax optimal confidence intervals. They compute this estimator and the associated confidence interval by approximating the saddle-point of a function. While this optimization problem is convex, it is rather difficult to solve using existing off-the-shelf optimization software. Furthermore, this computation can be expensi…
Generalized extreme value (GEV) regression is often more adapted when we investigate a relationship between a binary response variable $Y$ which represents a rare event and potentiel predictors $\mathbf{X}$. In particular, we use the quantile function of the GEV distribution as link function. Bootstrapping assigns measures of accuracy (bias, variance, confidence intervals, prediction error, test of hypothesis) to sample estimates. This technique allows estimation of the sampling distribution of almost any statistic using random sampling methods. Bootstrapping estimates the properties of an est…
We compare two ways of constructing confidence intervals for the moments-matching parameter estimates of a Gaussian spatio-temporal Ornstein-Uhlenbeck process. It was found that those obtained via pairwise likelihood approximations had lower coverages and were more prone to the curse of dimensionality as opposed to those from a parametric bootstrap procedure.
We construct robust empirical Bayes confidence intervals (EBCIs) in a normal means problem. The intervals are centered at the usual linear empirical Bayes estimator, but use a critical value accounting for shrinkage. Parametric EBCIs that assume a normal distribution for the means (Morris, 1983b) may substantially undercover when this assumption is violated. In contrast, our EBCIs control coverage regardless of the means distribution, while remaining close in length to the parametric EBCIs when the means are indeed Gaussian. If the means are treated as fixed, our EBCIs have an average coverage…
We present a (selective) review of recent frequentist high-dimensional inference methods for constructing $p$-values and confidence intervals in linear and generalized linear models. We include a broad, comparative empirical study which complements the viewpoint from statistical methodology and theory. Furthermore, we introduce and illustrate the R-package hdi which easily allows the use of different methods and supports reproducibility.
We develop confidence bounds that hold uniformly over time for off-policy evaluation in the contextual bandit setting. These confidence sequences are based on recent ideas from martingale analysis and are non-asymptotic, non-parametric, and valid at arbitrary stopping times. We provide algorithms for computing these confidence sequences that strike a good balance between computational and statistical efficiency. We empirically demonstrate the tightness of our approach in terms of failure probability and width and apply it to the "gated deployment" problem of safely upgrading a production conte…
The area under the curve (AUC) of summary receiver operating characteristic (SROC) curve is a primary statistical outcome for meta-analysis of diagnostic test accuracy studies (DTA). However, its confidence interval has not been reported in most of DTA meta-analyses, because no certain methods and statistical packages have been provided. In this article, we provide a bootstrap algorithm for computing the confidence interval of the AUC. Also, using the bootstrap framework, we can conduct a bootstrap test for assessing significance of the difference of AUCs for multiple diagnostic tests. In addi…
The conformalClassification package implements Transductive Conformal Prediction (TCP) and Inductive Conformal Prediction (ICP) for classification problems. Conformal Prediction (CP) is a framework that complements the predictions of machine learning algorithms with reliable measures of confidence. TCP gives results with higher validity than ICP, however ICP is computationally faster than TCP. The package conformalClassification is built upon the random forest method, where votes of the random forest for each class are considered as the conformity scores for each data point. Although the main …
Yale University（耶鲁大学）；University of Michigan（密歇根大学）
Information theoretic measures (e.g. the Kullback Liebler divergence and Shannon mutual information) have been used for exploring possibly nonlinear multivariate dependencies in high dimension. If these dependencies are assumed to follow a Markov factor graph model, this exploration process is called structure discovery. For discrete-valued samples, estimates of the information divergence over the parametric class of multinomial models lead to structure discovery methods whose mean squared error achieves parametric convergence rates as the sample size grows. However, a naive application of thi…
I present a frequentist method for quantifying uncertainty when correcting correlations for attenuation due to measurement error. The method is conservative but has far better coverage properties than the methods currently used when sample sizes are small. I recommend the use of confidence curves in favor of confidence intervals when this method is used. I introduce the R package "attenuation" which can be used to calculate and visualize the methods described in this paper.
The aim of this paper is to construct the confidence interval of the ultimate ruin probability under the insurance surplus driven by a Lévy process. Assuming a parametric family for the Lévy measures, we estimate the parameter from the surplus data and estimate the ruin probability via the delta method. However the asymptotic variance includes the derivative of the ruin probability with respect to the parameter, which is not generally given explicitly, and the confidence interval is not straightforward even if the ruin probability is well estimated. This paper gives the Cramér-type approximati…
Suppose one has a collection of parameters indexed by a (possibly infinite dimensional) set. Given data generated from some distribution, the objective is to estimate the maximal parameter in this collection evaluated at this distribution. This estimation problem is typically non-regular when the maximizing parameter is non-unique, and as a result standard asymptotic techniques generally fail in this case. We present a technique for developing parametric-rate confidence intervals for the quantity of interest in these non-regular settings. We show that our estimator is asymptotically efficient …
Multinomial processing tree (MPT) models are tools for disentangling the contributions of latent cognitive processes in a given experimental paradigm. The present note analyzes MPT models subject to order constraints on subsets of its parameters. The constraints that we consider frequently arise in cases where the response categories are ordered in some sense such as in confidence-rating data, Likert scale data, where graded guessing tendencies or response biases are created via base-rate or payoff manipulations, in the analysis of contingency tables with order constraints, and in many other c…
Confidence interval procedures used in low dimensional settings are often inappropriate for high dimensional applications. When a large number of parameters are estimated, marginal confidence intervals associated with the most significant estimates have very low coverage rates: They are too small and centered at biased estimates. The problem of forming confidence intervals in high dimensional settings has previously been studied through the lens of selection adjustment. In this framework, the goal is to control the proportion of non-covering intervals formed for selected parameters. In this pa…
Due to the increasing availability of high-dimensional empirical applications in many research disciplines, valid simultaneous inference becomes more and more important. For instance, high-dimensional settings might arise in economic studies due to very rich data sets with many potential covariates or in the analysis of treatment heterogeneities. Also the evaluation of potentially more complicated (non-linear) functional forms of the regression relationship leads to many potential variables for which simultaneous inferential statements might be of interest. Here we provide a review of classica…
Semi-competing risks refer to the setting where primary scientific interest lies in estimation and inference with respect to a non-terminal event, the occurrence of which is subject to a terminal event. In this paper, we present the R package SemiCompRisks that provides functions to perform the analysis of independent/clustered semi-competing risks data under the illness-death multi-state model. The package allows the user to choose the specification for model components from a range of options giving users substantial flexibility, including: accelerated failure time or proportional hazards re…
Effect size indices are useful parameters that quantify the strength of association and are unaffected by sample size. There are many available effect size parameters and estimators, but it is difficult to compare effect sizes across studies as most are defined for a specific type of population parameter. We recently introduced a new Robust Effect Size Index (RESI) and confidence interval, which is advantageous because it is not model-specific. Here we present the RESI R package, which makes it easy to report the RESI and its confidence interval for many different model classes, with a consist…
Researchers are often interested in using longitudinal data to estimate the causal effects of hypothetical time-varying treatment interventions on the mean or risk of a future outcome. Standard regression/conditioning methods for confounding control generally fail to recover causal effects when time-varying confounders are themselves affected by past treatment. In such settings, estimators derived from Robins's g-formula may recover time-varying treatment effects provided sufficient covariates are measured to control confounding by unmeasured risk factors. The package gfoRmula implements in R …
