arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.17031stat.ME
收藏

通过最优资源分配在实验丰富环境中最小化II类错误

Minimizing Type 2 Errors in an Experiment-Rich Regime via Optimal Resource Allocation

Stephen M. Ross School of Business, University of Michigan(密歇根大学罗斯商学院)

Fenghua Yang, Dae Woong Ham, Stefanus Jasin

AI总结:

本文研究在实验丰富环境下如何分配有限资源以最小化II类错误,提出通过修正因子提升检测效能的方法,理论分析与实验验证显示其在实践中具有竞争力。

摘要:

随机实验(通常称为

英文摘要:

Randomized experiments (often known as "A/B tests") are widely used to evaluate product and service innovations. We study how to allocate limited experimentation resources across M concurrent experiments in an experiment-rich regime. Existing work on allocation has predominantly focused on minimizing the worst-case mean squared error (MSE) of estimated treatment effects, which favors experiments with larger (and typically unknown) outcome variance. While appropriate for controlling estimation accuracy, this objective does not directly capture a common managerial priority in screening stages: detecting practically meaningful treatment effects with high probability. Motivated by this, we consider the objective of minimizing the worst-case Type II error across all experiments. When the standard deviations are known, we characterize the power-optimal allocation and show that MSE-based allocations can be highly inefficient for detection, even though the two objectives align asymptotically. When the standard deviations are unknown and must be learned from pilot data, we show that a naive plug-in approach, treating pilot standard deviations as truth, can suffer substantial power loss. We propose inflating pilot estimates via correction factors and develop three optimization-based frameworks for selecting them, each reflecting a different risk criterion with distinct managerial implications. Although the resulting stochastic programs are computationally challenging at scale, we derive tractable surrogate reformulations inspired by robust optimization and establish favorable theoretical properties. We further propose Surrogate-S, a fully data-dependent and implementable procedure that computes correction factors using only pilot variance estimates and achieves near-oracle performance in numerical experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑