arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.00712stat.ME
收藏

基于差值的亚组处理效应在线检验

Online Testing of Subgroup Treatment Effects Based on Value Difference

Miao Yu, Wenbin Lu, Rui Song

摘要:

在线A/B测试在高科技行业中起着指导产品开发和加速创新的关键作用。它执行原假设统计检验以确定哪个变体更好。然而,典型的A/B测试存在两个问题:(i)固定视野框架扩大了连续监测下的假阳性误差;(ii)同质效应假设无法识别具有有益处理效应的亚组。在本文中,我们提出了一个序贯测试的亚组处理效应的基础上的价值差异,命名为SUBTLE,同时解决这两个问题。SUBTLE允许实验者在实验过程中“偷看”结果,而不会损害统计保证。它假设异质性处理效应,旨在检验是否有部分人群亚组将从研究性治疗中获益。如果检测结果表明存在此类亚组,则将使用现成的估计最佳治疗规则识别亚组。我们研究了我们提出的测试模拟和真实数据集的实证性能。结果表明,与相应的固定时域检验相比,SUBTLE具有较高的检测能力,在任何时刻都能控制第一类错误,对噪声协变量具有更强的鲁棒性,并能实现早期停止.

英文摘要:

Online A/B testing plays a critical role in the high-tech industry to guide product development and accelerate innovation. It performs a null hypothesis statistical test to determine which variant is better. However, a typical A/B test presents two problems: (i) a fixed-horizon framework inflates the false-positive errors under continuous monitoring; (ii) the homogeneous effects assumption fails to identify a subgroup with a beneficial treatment effect. In this paper, we propose a sequential test for subgroup treatment effects based on value difference, named SUBTLE, to address these two problems simultaneously. The SUBTLE allows the experimenters to "peek" at the results during the experiment without harming the statistical guarantees. It assumes heterogeneous treatment effects and aims to test if some subgroup of the population will benefit from the investigative treatment. If the testing result indicates the existence of such a subgroup, a subgroup will be identified using a readily available estimated optimal treatment rule. We examine the empirical performance of our proposed test on both simulations and a real dataset. The results show that the SUBTLE has high detection power with controlled type I error at any time, is more robust to noise covariates, and can achieve early stopping compared with the corresponding fixed-horizon test.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑