arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1710.01285stat.ME
收藏

基于估计量序列的大规模Web实验分析

Analysis of Large Scale Web Experiments Using Sequences of Estimators

Ian E. Fellows

摘要:

实验测试在Web应用程序的优化中至关重要,因此A/B测试已被广泛采用作为确定许多Web应用程序的最佳内容的方法。虽然一些测试平台提供了顺序有效的推断,但大部分在线测试仍然使用传统的统计测试,这些测试不允许临时“窥视”数据或将测试扩展到超过其建议的样本量。在本文中,我们开发的结果有用的大规模实验的序贯分析。特别是,研究了极大似然序列和广义矩估计量方法的性质。这导致了对二元结果的比值比和相对风险的新测试。对于连续和有序的结果,我们开发了平均差检验和曲线下面积(AUC)的非参数检验。此外,提出了这些测试的多变量版本。

英文摘要:

Experimental testing is vital in the optimization of web applications, and as such A/B testing has been widely adopted as a methodology for determining optimal content for many web applications. While some testing platforms provide sequentially valid inferences, a large proportion of online tests still utilize traditional statistical tests that do not allow for interim "peeking" at the data or extending the test past its proposed sample size. In this paper we develop results useful for the sequential analysis of large scale experiments. In particular, the properties of sequences of maximum likelihood and generalized method of moments estimators are examined. This leads to new tests of odds ratios and relative risks for binary outcomes. For continuous and ordinal outcome we develop a test of mean difference and a non-parametric test of Area Under the Curve (AUC). Additionally, multivariate versions of these tests are proposed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑