在Booking.com上实现在线受控实验的民主化
Democratizing online controlled experiments at Booking.com
摘要:
有大量关于在线对照实验的文献,既有可用于分析实验结果的统计方法,也有几家大型互联网公司建立的基础设施,还有拥抱在线实验以告知产品开发的组织挑战。在Booking.com,我们已经使用在线实验进行了十多年的基于证据的产品开发。我们的方法和基础设施从一开始就被设计成反映Booking.com文化,也就是说,考虑到实验和决策的民主化和分散化。在本文中,我们解释了如何建立一个成功和失败的中央存储库,以允许知识共享,有一个通用的和可扩展的代码库,它强制执行实验和业务逻辑之间的松散耦合,密切和透明的监测质量和数据收集管道的可靠性,以建立信任的实验基础设施,并采取保障措施,使任何人都能够对他们的实验拥有端到端的所有权,这使得像www.example.com这样的大型组织Booking.com能够真正并成功地使实验民主化。
英文摘要:
There is an extensive literature about online controlled experiments, both on the statistical methods available to analyze experiment results as well as on the infrastructure built by several large scale Internet companies but also on the organizational challenges of embracing online experiments to inform product development. At Booking.com we have been conducting evidenced based product development using online experiments for more than ten years. Our methods and infrastructure were designed from their inception to reflect Booking.com culture, that is, with democratization and decentralization of experimentation and decision making in mind. In this paper we explain how building a central repository of successes and failures to allow for knowledge sharing, having a generic and extensible code library which enforces a loose coupling between experimentation and business logic, monitoring closely and transparently the quality and the reliability of the data gathering pipelines to build trust in the experimentation infrastructure, and putting in place safeguards to enable anyone to have end to end ownership of their experiments have allowed such a large organization as Booking.com to truly and successfully democratize experimentation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。