鸽巢设计：从在线匹配视角平衡序贯实验
Pigeonhole Design: Balancing Sequential Experiments from an Online Matching Perspective
AI总结：
针对在线A/B测试中协变量平衡难题，提出鸽巢设计，通过划分协变量空间并逐鸽巢平衡，显著降低估计方差，优于配对与完全随机设计。
摘要：
实践者和学者长期以来都认识到在开展随机实验时协变量平衡的益处。然而，对于进行在线A/B测试的网络公司而言，当实验对象按顺序到达时，平衡协变量信息仍然具有挑战性。在本文中，我们研究了一个在线实验设计问题，我们称之为“在线阻塞问题”。在该问题中，具有异质协变量信息的实验对象依次到达，并且必须立即被分配到对照组或处理组。目标是最小化总差异，该差异定义为两组之间的最小权重完美匹配。为了解决这个问题，我们提出了一种随机实验设计，我们称之为“鸽巢设计”。鸽巢设计首先将协变量空间划分为更小的空间，我们称之为鸽巢，然后，当实验对象到达每个鸽巢时，平衡每个鸽巢的对照组和处理组数量。我们分析了鸽巢设计的理论性能，并通过与两个著名的基准设计（配对设计和完全随机设计）进行比较来展示其有效性。我们确定了鸽巢设计比基准设计展现出更多优势的场景。最后，我们使用Yahoo!数据进行了大量模拟，结果表明，如果使用鸽巢设计来估计平均处理效应，方差可减少10.2%。
英文摘要：
Practitioners and academics have long appreciated the benefits of covariate balancing when they conduct randomized experiments. For web-facing firms running online A/B tests, however, it still remains challenging in balancing covariate information when experimental subjects arrive sequentially. In this paper, we study an online experimental design problem, which we refer to as the "Online Blocking Problem." In this problem, experimental subjects with heterogeneous covariate information arrive sequentially and must be immediately assigned into either the control or the treated group. The objective is to minimize the total discrepancy, which is defined as the minimum weight perfect matching between the two groups. To solve this problem, we propose a randomized design of experiment, which we refer to as the "Pigeonhole Design." The pigeonhole design first partitions the covariate space into smaller spaces, which we refer to as pigeonholes, and then, when the experimental subjects arrive at each pigeonhole, balances the number of control and treated subjects for each pigeonhole. We analyze the theoretical performance of the pigeonhole design and show its effectiveness by comparing against two well-known benchmark designs: the match-pair design and the completely randomized design. We identify scenarios when the pigeonhole design demonstrates more benefits over the benchmark design. To conclude, we conduct extensive simulations using Yahoo! data to show a 10.2% reduction in variance if we use the pigeonhole design to estimate the average treatment effect.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
