arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.02374cs.SE
收藏

仿真驱动的自动化端到端测试和Oracle推理

Simulation-Driven Automated End-to-End Test and Oracle Inference

Shreshth Tuli, Kinga Bojarczuk, Natalija Gucevska, Mark Harman, Xiao-Yu Wang, Graham Wright

摘要:

这是第一个工作报告推理测试在工业规模。具体来说,它报告了Meta的完整性系统自动化测试的经验。我们构建了一个名为ALPACAS的内部工具,用于端到端完整性测试的自动推理。完整性测试旨在通过检查在平台上发生有害行为时是否会进行干预来确保用户在线安全。ALPACAS不仅推断测试输入,而且还通过观察生产干预来推断神谕,以防止有害行为。这种方法允许Meta自动化为其平台(如Facebook和Instagram)生成完整性测试的过程,这些平台由数亿行生产代码组成。我们概述了ALPACAS的设计和部署,并报告其覆盖率,在测试推理过程的每个阶段产生的测试数量,以及它们的通过率。具体来说,我们证明了使用ALPACAS显着提高覆盖率从手动测试设计的完整性的特定方面的端到端的测试,它被应用到。此外,从300万个数据点中,ALPACAS自动生成39个生产就绪的端到端完整性测试。我们还报告说,ALPACAS推断的测试套件在端到端测试中具有非常低的片状性,其平均生产通过率为99.84%。

英文摘要:

This is the first work to report on inferential testing at scale in industry. Specifically, it reports the experience of automated testing of integrity systems at Meta. We built an internal tool called ALPACAS for automated inference of end-to-end integrity tests. Integrity tests are designed to keep users safe online by checking that interventions take place when harmful behaviour occurs on a platform. ALPACAS infers not only the test input, but also the oracle, by observing production interventions to prevent harmful behaviour. This approach allows Meta to automate the process of generating integrity tests for its platforms, such as Facebook and Instagram, which consist of hundreds of millions of lines of production code. We outline the design and deployment of ALPACAS, and report results for its coverage, number of tests produced at each stage of the test inference process, and their pass rates. Specifically, we demonstrate that using ALPACAS significantly improves coverage from a manual test design for the particular aspect of integrity end-to-end testing it was applied to. Further, from a pool of 3 million data points, ALPACAS automatically yields 39 production-ready end-to-end integrity tests. We also report that the ALPACAS-inferred test suite enjoys exceptionally low flakiness for end-to-end testing with its average in-production pass rate of 99.84%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑