For classification problems with significant class imbalance, subsampling can reduce computational costs at the price of inflated variance in estimating model parameters. We propose a method for subsampling efficiently for logistic regression by adjusting the class balance locally in feature space via an accept-reject scheme. Our method generalizes standard case-control sampling, using a pilot estimate to preferentially select examples whose responses are conditionally rare given their features. The biased subsampling is corrected by a post-hoc analytic adjustment to the parameters. The method is simple and requires one parallelizable scan over the full data set. Standard case-control sampling is inconsistent under model misspecification for the population risk-minimizing coefficients $θ^*$. By contrast, our estimator is consistent for $θ^*$ provided that the pilot estimate is. Moreover, under correct specification and with a consistent, independent pilot estimate, our estimator has exactly twice the asymptotic variance of the full-sample MLE - even if the selected subsample comprises a miniscule fraction of the full data set, as happens when the original data are severely imbalanced. The factor of two improves to $1+\frac{1}{c}$ if we multiply the baseline acceptance probabilities by $c>1$ (and weight points with acceptance probability greater than 1), taking roughly $\frac{1+c}{2}$ times as many data points into the subsample. Experiments on simulated and real data show that our method can substantially outperform standard case-control subsampling.
More Efficient Estimation for Logistic Regression with Optimal Subsample
本文提出了一种改进的逻辑斯谛回归估计方法，该方法基于Wang et al. 2018中提出的最优子采样概率所取的子样本。渐近结果和数值结果都表明，新的估计量具有更高的估计效率。我们还开发了一种新的算法的基础上泊松子采样，它不需要近似的最佳子采样概率一次。当可用的随机存取存储器不足以保存全部数据时，这在计算上是有利的。有趣的是，渐近分布还表明，如果采样率（子样本量与完整数据样本量的比率）不收敛于零，则泊松子采样会产生更有效的估计量。我们还得到了基于Poisson子抽样的估计量的无条件渐近分布。所提出的方法需要使用一个导频估计器来校正未加权估计器的偏差。我们进一步表明，即使导频估计是不一致的，所得到的估计仍然是一致的和渐近正态的，如果该模型是正确指定的。
Stabilised weighted data subsampling for accelerated Bayesian inference in models with recursive likelihoods
University of Technology Sydney（悉尼科技大学）；University of New South Wales（新南威尔士大学）；Human Technology Institute, University of Technology Sydney（悉尼科技大学人类技术研究所）
