arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.10470cs.LGcs.IR
收藏

Adyen支付场景下的离线策略评估

Off-policy Evaluation for Payments at Adyen

Adyen(阿迪扬)

Alex Egg

AI总结:

本文介绍Adyen将离线策略评估(OPE)用于推荐系统优化,以十亿级交易数据验证其与A/B测试结果强相关,预计六个月内带来900万至5400万笔增量交易,并探讨了生产环境中的实践挑战与估计器选择。

摘要:

本文展示了离线策略评估(OPE)在Adyen(全球领先的金融支付处理公司)中成功应用于加速推荐系统开发和优化的实践。针对传统A/B测试速度慢、成本高且往往无法得出明确结论的局限性,我们整合了OPE,利用历史数据对新推荐系统变体进行快速评估。我们在十亿级交易数据集上进行的分析显示,OPE估计值与在线A/B测试结果之间存在强相关性,预计在六个月内可带来900万至5400万笔的增量交易。我们探讨了在高吞吐量生产环境中部署OPE所面临的实际挑战和权衡,包括利用探索流量进行数据收集、降低重要性采样中的方差,以及通过使用Apache Spark确保可扩展性。通过对各种OPE估计器进行基准测试,我们为它们在大规模工业支付系统决策中的有效性及集成提供了指导。

英文摘要:

This paper demonstrates the successful application of Off-Policy Evaluation (OPE) to accelerate recommender system development and optimization at Adyen, a global leader in financial payment processing. Facing the limitations of traditional A/B testing, which proved slow, costly, and often inconclusive, we integrated OPE to enable rapid evaluation of new recommender system variants using historical data. Our analysis, conducted on a billion-scale dataset of transactions, reveals a strong correlation between OPE estimates and online A/B test results, projecting an incremental 9--54 million transactions over a six-month period. We explore the practical challenges and trade-offs associated with deploying OPE in a high-volume production environment, including leveraging exploration traffic for data collection, mitigating variance in importance sampling, and ensuring scalability through the use of Apache Spark. By benchmarking various OPE estimators, we provide guidance on their effectiveness and integration into the decision-making systems for large-scale industrial payment systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑