arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.01977stat.MLcs.LG
收藏

为什么适应性收集的数据具有负偏差以及如何纠正它

Why Adaptively Collected Data Have Negative Bias and How to Correct for It

Stanford University(斯坦福大学)

Xinkun Nie, Xiaoying Tian, Jonathan Taylor, James Zou

摘要:

从科学实验到在线A/B测试,先前观察到的数据往往会影响未来实验的执行方式,进而影响将收集哪些数据。这种适应性在数据和收集程序之间引入了复杂的关联。本文证明了当数据采集过程满足自然条件时,数据的样本均值具有系统性\emph{negative}偏差。作为一个例子,考虑一个适应性临床试验,其中更多的数据点更有可能被测试用于显示初步前景的治疗。我们令人惊讶的结果意味着观察到的平均处理效应会低估每种处理的真实效果。我们定量地分析了在各种环境中这种负偏差的大小和行为。我们还提出了一种新的基于选择性推理技术的去偏置算法。实验结果表明,该方法能有效地减小偏差和估计误差。

英文摘要:

From scientific experiments to online A/B testing, the previously observed data often affects how future experiments are performed, which in turn affects which data will be collected. Such adaptivity introduces complex correlations between the data and the collection procedure. In this paper, we prove that when the data collection procedure satisfies natural conditions, then sample means of the data have systematic \emph{negative} biases. As an example, consider an adaptive clinical trial where additional data points are more likely to be tested for treatments that show initial promise. Our surprising result implies that the average observed treatment effects would underestimate the true effects of each treatment. We quantitatively analyze the magnitude and behavior of this negative bias in a variety of settings. We also propose a novel debiasing algorithm based on selective inference techniques. In experiments, our method can effectively reduce bias and estimation error.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑