Modern statistics provides an ever-expanding toolkit for estimating unknown parameters. Consequently, applied statisticians frequently face a difficult decision: retain a parameter estimate from a familiar method or replace it with an estimate from a newer or more complex one. While it is traditional to compare estimates using risk, such comparisons are rarely conclusive in realistic settings.
In response, we propose the "c-value" as a measure of confidence that a new estimate achieves smaller loss than an old estimate on a given dataset. We show that it is unlikely that a large c-value coincides with a larger loss for the new estimate. Therefore, just as a small p-value supports rejecting a null hypothesis, a large c-value supports using a new estimate in place of the old. For a wide class of problems and estimates, we show how to compute a c-value by first constructing a data-dependent high-probability lower bound on the difference in loss. The c-value is frequentist in nature, but we show that it can provide validation of shrinkage estimates derived from Bayesian models in real data applications involving hierarchical models and Gaussian processes.
When evaluating and comparing models using leave-one-out cross-validation (LOO-CV), the uncertainty of the estimate is typically assessed using the variance of the sampling distribution. Considering the uncertainty is important, as the variability of the estimate can be high in some cases. An important result by Bengio and Grandvalet (2004) states that no general unbiased variance estimator can be constructed, that would apply for any utility or loss measure and any model. We show that it is possible to construct an unbiased estimator considering a specific predictive performance measure and m…
Comparing Scale Parameter Estimators for Gaussian Process Interpolation with the Brownian Motion Prior: Leave-One-Out Cross Validation and Maximum Likelihood
In this article, we present the maximum weighted likelihood estimator (MWLE) for robust estimations of heavy-tail finite mixture models (FMM). This is motivated by the complex distributional phenomena of insurance claim severity data, where flexible density estimation tools such as FMM are needed but MLE often produces unstable tail estimates under FMM. Under some regularity conditions, MWLE is proved to be consistent and asymptotically normal. We further prove that the tail index obtained by MWLE is consistent even if the model is misspecified, justifying the robustness of MWLE in estimating …
In this paper we deal with the regression problem in a random design setting. We investigate asymptotic optimality under minimax point of view of various Bayesian rules based on warped wavelets and show that they nearly attain optimal minimax rates of convergence over the Besov smoothness class considered. Warped wavelets have been introduced recently, they offer very good computable and easy-to-implement properties while being well adapted to the statistical problem at hand. We particularly put emphasis on Bayesian rules leaning on small and large variance Gaussian priors and discuss their si…
We consider the problem of efficient statistical inference for comparing two regression curves estimated from two samples of dependent measurements. Based on a representation of the best pair of linear unbiased estimators in continuous time models as a stochastic integral, an efficient pair of linear unbiased estimators with corresponding optimal designs for finite sample size is constructed. This pair minimises the width of the confidence band for the difference between the estimated curves. We thus extend results readily available in the literature to the case of correlated observations and …
Inference procedures in sequential trial emulation with survival outcomes: comparing confidence intervals based on the sandwich variance estimator, bootstrap and jackknife
There is a long-standing debate in the statistical, epidemiological and econometric fields as to whether nonparametric estimation that uses data-adaptive methods, like machine learning algorithms in model fitting, confer any meaningful advantage over simpler, parametric approaches in real-world, finite sample estimation of causal effects. We address the question: when trying to estimate the effect of a treatment on an outcome, across a universe of reasonable data distributions, how much does the choice of nonparametric vs.~parametric estimation matter? Instead of answering this question with s…
The problem of estimating a high-dimensional sparse vector $\boldsymbolθ \in \mathbb{R}^n$ from an observation in i.i.d. Gaussian noise is considered. The performance is measured using squared-error loss. An empirical Bayes shrinkage estimator, derived using a Bernoulli-Gaussian prior, is analyzed and compared with the well-known soft-thresholding estimator. We obtain concentration inequalities for the Stein's unbiased risk estimate and the loss function of both estimators. The results show that for large $n$, both the risk estimate and the loss function concentrate on deterministic values clo…
Sensitivity Analysis and Optimization of Stochastic Epidemic Models under Parameter Uncertainty
University of Maryland, College Park（马里兰大学帕克分校）；Robert H. Smith School of Business, University of Maryland, College Park（马里兰大学罗伯特H史密斯商学院）；Institute for Systems Research, University of Maryland, College Park（马里兰大学系统研究所）
University of Fribourg（弗里堡大学）；Swiss Institute of Bioinformatics（瑞士生物信息学研究所）；University of Gothenburg（哥德堡大学）；Gothenburg Global Biodiversity Center（哥德堡全球生物多样性中心）
Deep Bayesian neural networks (BNNs) are a powerful tool, though computationally demanding, to perform parameter estimation while jointly estimating uncertainty around predictions. BNNs are typically implemented using arbitrary normal-distributed prior distributions on the model parameters. Here, we explore the effects of different prior distributions on classification tasks in BNNs and evaluate the evidence supporting the predictions based on posterior probabilities approximated by Markov Chain Monte Carlo sampling and by computing Bayes factors. We show that the choice of priors has a substa…
Conditional simulation via entropic optimal transport: Toward non-parametric estimation of conditional Brenier maps
CalTech（加州理工学院）；Center for Data Science, New York University（纽约大学数据科学中心）；Massachusets Institute of Technology（麻省理工学院）；Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）
We consider the problem of designing experiments for the comparison of two regression curves describing the relation between a predictor and a response in two groups, where the data between and within the group may be dependent. In order to derive efficient designs we use results from stochastic analysis to identify the best linear unbiased estimator (BLUE) in a corresponding continuous time model. It is demonstrated that in general simultaneous estimation using the data from both groups yields more precise results than estimation of the parameters separately in the two groups. Using the BLUE …
In the last years there has been a growing interest in proposing methods for estimating covariance functions for geostatistical data. Among these, maximum likelihood estimators have nice features when we deal with a Gaussian model. However maximum likelihood becomes impractical when the number of observations is very large. In this work we review some solutions and we contrast them in terms of loss of statistical efficiency and computational burden. Specifically we focus on three types of weighted composite likelihood functions based on pairs and we compare them with the method of covariance t…
Consider measured positions of the paleomagnetic north pole over time. Each measured position may be viewed as a direction, expressed as a unit vector in three dimensions and incorporating some error. In this sequence, the true directions are expected to be close to one another at nearby times. A simple trend estimator that respects the geometry of the sphere is to compute a running average over the time-ordered observed direction vectors, then normalize these average vectors to unit length. This paper treats a considerably richer class of competing directional trend estimators that respect sp…
This paper investigates the use of bootstrap-based bias correction of semi-parametric estimators of the long memory parameter in fractionally integrated processes. The re-sampling method involves the application of the sieve bootstrap to data pre-filtered by a preliminary semi-parametric estimate of the long memory parameter. Theoretical justification for using the bootstrap techniques to bias adjust log-periodogram and semi-parametric local Whittle estimators of the memory parameter is provided. Simulation evidence comparing the performance of the bootstrap bias correction with analytical bia…
This paper investigates bootstrap-based bias correction of semiparametric estimators of the long memory parameter, $d$, in fractionally integrated processes. The re-sampling method involves the application of the sieve bootstrap to data pre-filtered by a preliminary semiparametric estimate of the long memory parameter. Theoretical justification for using the bootstrap technique to bias adjust log periodogram and semiparametric local Whittle estimators of the memory parameter is provided in the case where the true value of $d$ lies in the range $0\leq d<0.5$. That the bootstrap method provides …
Randomly censored survival data are frequently encountered in applied sciences including biomedical or reliability applications and clinical trial analyses. Testing the significance of statistical hypotheses is crucial in such analyses to get conclusive inference but the existing likelihood based tests, under a fully parametric model, are extremely non-robust against outliers in the data. Although, there exists a few robust parameter estimators (e.g., M-estimators and minimum density power divergence estimators) given randomly censored data, there is hardly any robust testing procedure availab…
Propensity score (PS) based estimators are increasingly used for causal inference in observational studies. However, model selection for PS estimation in high-dimensional data has received little attention. In these settings, PS models have traditionally been selected based on the goodness-of-fit for the treatment mechanism itself, without consideration of the causal parameter of interest. Collaborative minimum loss-based estimation (C-TMLE) is a novel methodology for causal inference that takes into account information on the causal parameter of interest when selecting a PS model. This "colla…
