arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.10331stat.ME
收藏

综合证据:数据池化作为在线实验中治疗选择的工具

Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments

The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学)

Zhenkang Peng, Chengzhang Li, Ying Rong, Renyu Zhang

AI总结:

提出数据池化治疗推出(DPTR)框架,通过跨实验池化数据减少估计变异性,支持重叠与非重叠流量场景,理论分析和实证验证其优于传统方法。

摘要:

随机实验是因果推断的金标准,但在商业应用中面临重大挑战,包括有限的流量分配、异质性治疗效果估计的需求以及管理重叠实验的复杂性。这些因素导致治疗效果估计的高变异性,使得数据驱动的政策推出变得困难。为了解决这些问题,我们引入了数据池化治疗推出(DPTR)框架,该框架通过跨实验池化数据而非狭隘地关注单个实验来增强政策推出。DPTR 能有效适应重叠和非重叠流量场景,无论模型规格是线性还是非线性。我们通过三管齐下的验证证明了该框架的稳健性:(a) 理论分析表明,在非重叠实验下,特别是当实验数量较大时,DPTR 优于传统的均值差和普通最小二乘法;(b) 合成模拟证实了其在具有重叠流量、丰富协变量和非线性规格的复杂场景中的适应性;(c) 对来自真实世界平台的两个实验数据集进行实证应用,展示了其在指导单个实验内子组的定制政策推出以及协调多个重叠实验场景中的政策部署方面的有效性。通过减少估计变异性以提高决策有效性,DPTR 为在线平台在当今日益复杂的商业环境中更好地利用其实验数据提供了一种可扩展、实用的解决方案。

英文摘要:

Randomized experiments are the gold standard for causal inference but face significant challenges in business applications, including limited traffic allocation, the need for heterogeneous treatment effect estimation, and the complexity of managing overlapping experiments. These factors lead to high variability in treatment effect estimates, making data-driven policy roll out difficult. To address these issues, we introduce the data pooling treatment roll-out (DPTR) framework, which enhances policy roll-out by pooling data across experiments rather than focusing narrowly on individual ones. DPTR can effectively accommodate both overlapping and non-overlapping traffic scenarios, regardless of linear or nonlinear model specifications. We demonstrate the framework's robustness through a three-pronged validation: (a) theoretical analysis shows that DPTR surpasses the traditional difference-in-mean and ordinary least squares methods under non-overlapping experiments, particularly when the number of experiments is large; (b) synthetic simulations confirm its adaptability in complex scenarios with overlapping traffic, rich covariates and nonlinear specifications; and (c) empirical applications to two experimental datasets from real world platforms, demonstrating its effectiveness in guiding customized policy roll-outs for subgroups within a single experiment, as well as in coordinating policy deployments across multiple experiments with overlapping scenarios. By reducing estimation variability to improve decision-making effectiveness, DPTR provides a scalable, practical solution for online platforms to better leverage their experimental data in today's increasingly complex business environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑