arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01443cs.AI
收藏

SimGym:基于流量的浏览器代理用于电子商务中的离线A/B测试

SimGym: Traffic-Grounded Browser Agents for Offline A/B Testing in E-Commerce

Shopify
Alberto Castelo, Zahra Zanjani Foumani, Ailin Fan, Keat Yang Koay, Vibhor Malik, Yuanzheng Zhu, Han Li, Meysam Feghhi, Ronie Uliana, Shuang Xie, Zhaoyu Zhang, A… 展开作者

Alberto Castelo, Zahra Zanjani Foumani, Ailin Fan, Keat Yang Koay, Vibhor Malik, Yuanzheng Zhu, Han Li, Meysam Feghhi, Ronie Uliana, Shuang Xie, Zhaoyu Zhang, Angelo Ocana Martins, Mingyu Zhao, Francis Pelland, Jonathan Faerman, Nikolas LeBlanc, Aaron Glazer, Andrew McNamara, Lingyun Wang, Zhong Wu

AI总结:

SimGym通过基于流量的合成买家和大型语言模型代理,实现快速离线A/B测试,提升电子商务实验效率

摘要:

A/B测试仍然是评估电子商务UI变更的黄金标准,但其分流流量、需要数周才能达到显著性,并可能损害用户体验。我们介绍了SimGym,一个可扩展的系统,利用由大型语言模型代理驱动的基于流量的合成买家进行快速离线A/B测试。SimGym从生产交互数据中提取每个店铺的买家资料和意图,识别出不同的行为类型,并在控制组和处理组的店铺上模拟加权会话。我们验证了SimGym与真实人类结果在真实UI变更上的表现,且在控制混杂因素的情况下。即使在训练后没有对齐,SimGym代理也能达到与观察到的成果变化相匹配的水平,并将实验周期从数周减少到不到一小时,从而在不暴露给真实买家的情况下实现快速实验。

英文摘要:

A/B testing remains the gold standard for evaluating e-commerce UI changes, yet it diverts traffic, takes weeks to reach significance, and risks harming user experience. We introduce SimGym, a scalable system for rapid offline A/B testing using traffic-grounded synthetic buyers powered by Large Language Model agents operating in a live browser. SimGym extracts per-shop buyer profiles and intents from production interaction data, identifies distinct behavioral archetypes, and simulates cohort-weighted sessions across control and treatment storefronts. We validate SimGym against real human outcomes from real UI changes on a major e-commerce platform under confounder control. Even without alignment post training, SimGym agents achieve state of the art alignment with observed outcome shifts and reduces experiment cycles from weeks to under an hour , enabling rapid experimentation without exposure to real buyers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑