提高 A/B 测试的敏感性：将 CUPED 与截尾均值技术相结合
Improving Sensitivity in A/B Tests: Integrating CUPED with Trimmed Mean Techniques
AI总结：
本文提出融合 CUPED 方差缩减与 Yuen t 检验截尾均值的方法，以提升重尾、偏态在线 A/B 指标的检验敏感性，并在更小样本下保持统计功效。
摘要：
在零膨胀且偏态的指标下，准确估计在线 A/B 测试中的处理效应颇具挑战。Welch t 检验等传统检验由于依赖均值而非百分位数等统计量，在重尾数据上往往缺乏敏感性。Controlled Experiments Using Pre-experiment Data（CUPED，利用实验前数据的受控实验）技术通过降低方差来提高敏感性，但对于高度偏态的指标，这种方差缩减仍不充分。另一方面，Yuen t 检验使用截尾均值来稳健地处理异常值和偏态。本文提出一种将 CUPED 的方差缩减与 Yuen t 检验的稳健性相结合的方法，以增强假设检验的敏感性。我们的新方法以有原则的方式整合截尾数据，提供了一个在方差缩减与稳健位置度量之间取得平衡的框架。我们证明，该方法能够在更小样本量下改进对显著效应的检测，从而在不损失统计功效的情况下实现更快的实验决策。本工作拓展了受控实验在以高度偏态或高方差数据为特征的环境中的实用性。
英文摘要：
Accurate estimation of treatment effects in online A/B testing is challenging with zero-inflated and skewed metrics. Traditional tests, like Welch's t-test, often lack sensitivity with heavy-tailed data due to their reliance on means, as opposed to e.g., percentiles. The Controlled Experiments Using Pre-experiment Data (CUPED) technique improves sensitivity by reducing variance, yet that variance reduction is insufficient for highly skewed metrics. Alternatively, Yuen's t-test uses trimmed means to robustly handle outliers and skewness. This paper introduces a method that combines the variance reduction of CUPED with the robustness of Yuen's t-test to enhance hypothesis testing sensitivity. Our novel approach integrates trimmed data in a principled manner, offering a framework that balances variance reduction with robust location measures. We demonstrate improved detection of significant effects with smaller sample sizes, enabling quicker experimental decisions without sacrificing statistical power. This work broadens the utility of controlled experiments in environments characterized by highly skewed or high-variance data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
