SimAB:基于人格条件化AI代理的A/B测试模拟用于快速设计评估
SimAB: Simulating A/B Tests with Persona-Conditioned AI Agents for Rapid Design Evaluation
AI总结:
SimAB通过人格条件化AI代理模拟A/B测试,实现快速设计评估,准确率达67%,在高置信度案例中提升至83%。
摘要:
A/B测试是验证设计决策的标准方法,但其依赖真实用户流量限制了迭代速度,并使某些实验不切实际。我们提出了SimAB,一个将A/B测试重新框架为快速、隐私保护的模拟系统,使用人格条件化AI代理。给定设计截图和转化目标,SimAB生成用户人格,部署它们作为代理以表达偏好,汇总结果并合成理由。通过与实验实践者进行形成性研究,我们识别了交通限制阻碍测试的场景,包括低流量页面、多变量比较、微优化和隐私敏感情境。我们的设计强调速度、早期反馈、可操作的理由和受众指定。我们评估SimAB与47个已知结果的历史A/B测试,总体准确率为67%,在高置信度案例中增加到83%。额外实验显示其对命名和位置偏见的鲁棒性,并展示了从人格中获得的准确性提升。实践者反馈表明,SimAB支持更快的评估周期和快速筛选传统A/B测试难以评估的设计。
英文摘要:
A/B testing is a standard method for validating design decisions, yet its reliance on real user traffic limits iteration speed and makes certain experiments impractical. We present SimAB, a system that reframes A/B testing as a fast, privacy-preserving simulation using persona-conditioned AI agents. Given design screenshots and a conversion goal, SimAB generates user personas, deploys them as agents that state their preference, aggregates results, and synthesizes rationales. Through a formative study with experimentation practitioners, we identified scenarios where traffic constraints hinder testing, including low-traffic pages, multi-variant comparisons, micro-optimizations, and privacy-sensitive contexts. Our design emphasizes speed, early feedback, actionable rationales, and audience specification. We evaluate SimAB against 47 historical A/B tests with known outcomes, achieving 67% overall accuracy, increasing to 83% for high-confidence cases. Additional experiments show robustness to naming and positional bias and demonstrate accuracy gains from personas. Practitioner feedback suggests that SimAB supports faster evaluation cycles and rapid screening of designs difficult to assess with traditional A/B tests.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。