arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.11070stat.ME
收藏

二分实验的可扩展分析

Scalable Analysis of Bipartite Experiments

Liang Shi, Edvard Bakhitov, Kenneth Hung, Brian Karrer, Charlie Walker, Monica Bhole, Okke Schrijvers

AI总结:

本文针对大型在线平台二分实验的推断问题,提出了协变量调整的 CA-ERL 估计器、基于随机化的推断方法及线性时间算法,在保证可扩展性的同时显著降低了方差和置信区间宽度。

摘要:

二分实验是一种随机实验,其中处理应用于一组单元(随机化单元),这组单元不同于分析单元,并且随机化单元与分析单元通过二分图相连。大型在线平台的实验规模要求在存在大型二分干扰图的情况下进行准确推断,以及高度可扩展的实现。在本文中,我们描述了新的推断方法,能够对二分实验进行实用、可扩展的分析:(1) 我们提出了 CA-ERL,即暴露重加权线性(ERL)估计器 [9] 的协变量调整变体,它在经验上产生了 60-90% 的方差缩减。(2) 我们引入了一种基于随机化的推断方法,并在图稀疏性假设下证明了 Wald 型置信区间的渐近有效性。(3) 我们提出了一种用于 CA-ERL 估计器随机化推断的线性时间算法,该算法可以在 Presto 或 Spark 等查询引擎中轻松实现。我们在 Meta 的一项真实实验中评估了我们的方法,该实验在 Facebook Groups 上随机化处理并分析了用户级指标,以及在合成数据上进行了模拟。真实世界数据表明,我们的 CA-ERL 估计器在实际环境中将置信区间(CI)宽度减少了 60-90%(相比于 ERL)。使用合成数据的模拟表明,我们的随机化推断程序在各实例中实现了正确的覆盖范围,而 ERL 估计器对于具有较大真实效应大小的实例具有错误地较小的 CI 宽度,并且在二分图稠密时过于保守。

英文摘要:

Bipartite Experiments are randomized experiments where the treatment is applied to a set of units (randomization units) that is different from the units of analysis, and randomization units and analysis units are connected through a bipartite graph. The scale of experimentation at large online platforms necessitates both accurate inference in the presence of a large bipartite interference graph, as well as a highly scalable implementation. In this paper, we describe new methods for inference that enable practical, scalable analysis of bipartite experiments: (1) We propose CA-ERL, a covariate-adjusted variant of the exposure-reweighted-linear (ERL) estimator [9], which empirically yields 60-90% variance reduction. (2) We introduce a randomization-based method for inference and prove asymptotic validity of a Wald-type confidence interval under graph sparsity assumptions. (3) We present a linear-time algorithm for randomization inference of the CA-ERL estimator, which can be easily implemented in query engines like Presto or Spark. We evaluate our methods both on a real experiment at Meta that randomized treatment on Facebook Groups and analyzed user-level metrics, as well as simulations on synthetic data. The real-world data shows that our CA-ERL estimator reduces the confidence interval (CI) width by 60-90% (compared to ERL) in a practical setting. The simulations using synthetic data show that our randomization inference procedure achieves correct coverage across instances, while the ERL estimator has incorrectly small CI widths for instances with large true effect sizes and is overly conservative when the bipartite graph is dense.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑