Bias-Variance Tradeoffs: Novel Applications

Doubly Robust Stein-Kernelized Monte Carlo Estimator: Simultaneous Bias-Variance Reduction and Supercanonical Convergence

Standard Monte Carlo computation is widely known to exhibit a canonical square-root convergence speed in terms of sample size. Two recent techniques, one based on control variate and one on importance sampling, both derived from an integration of reproducing kernels and Stein's identity, have been proposed to reduce the error in Monte Carlo computation to supercanonical convergence. This paper presents a more general framework to encompass both techniques that is especially beneficial when the sample generator is biased and noise-corrupted. We show our general estimator, which we call the doub…

The Bias-Variance Tradeoff of Doubly Robust Estimator with Targeted $L_1$ regularized Neural Networks Predictions

The Doubly Robust (DR) estimation of ATE can be carried out in 2 steps, where in the first step, the treatment and outcome are modeled, and in the second step the predictions are inserted into the DR estimator. The model misspecification in the first step has led researchers to utilize Machine Learning algorithms instead of parametric algorithms. However, existence of strong confounders and/or Instrumental Variables (IVs) can lead the complex ML algorithms to provide perfect predictions for the treatment model which can violate the positivity assumption and elevate the variance of DR estimator…

有限样本偏差方差权衡与纳入试验参与变量的因果森林模型以确保可推广性

Finite-sample bias-variance tradeoff with variables related to trial participation inserted into causal forest models for ensuring generalizability

本文研究了在因果森林模型中纳入与试验参与相关的变量对有限样本偏差方差权衡的影响，发现IPW方法能有效提升性能，强调单独处理选择偏差的重要性。

Bias Variance Tradeoff in Analysis of Online Controlled Experiments

Many organizations utilize large-scale online controlled experiments (OCEs) to accelerate innovation. Having high statistical power to detect small differences between control and treatment accurately is critical, as even small changes in key metrics can be worth millions of dollars or indicate user dissatisfaction for a very large number of users. For large-scale OCE, the duration is typically short (e.g. two weeks) to expedite changes and improvements to the product. In this paper, we examine two common approaches for analyzing usage data collected from users within the time window of an exp…

A bootstrap method for estimating bias and variance in statistical multispecies models using highly disparate data sets

Statistical multispecies models of multiarea marine ecosystems use a variety of data sources to estimate parameters using composite or weighted likelihood functions with associated weighting issues and questions on how to obtain variance estimates. Regardless of the method used to obtain point estimates, a method is needed for variance estimation. A bootstrap technique is introduced for the evaluation of uncertainty in such models, taking into account inherent spatial and temporal correlations in the data sets thus avoiding many model--specification issues, which are commonly transferred as as…

Designing Experiments Informed by Observational Studies

The increasing availability of passively observed data has yielded a growing methodological interest in "data fusion." These methods involve merging data from observational and experimental sources to draw causal conclusions -- and they typically require a precarious tradeoff between the unknown bias in the observational dataset and the often-large variance in the experimental dataset. We propose an alternative approach to leveraging observational data, which avoids this tradeoff: rather than using observational data for inference, we use it to design a more efficient experiment. We consider t…

粒子物理实验中效率估计量的偏差、方差与置信区间

Bias, variance, and confidence intervals for efficiency estimators in particle physics experiments

本文研究粒子物理实验中效率估计的偏差、方差和置信区间，针对泊松试验次数、加权样本及超泊松方差等特殊条件，提出广义Wilson区间并分析其覆盖率。