arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.11550cs.SE
收藏

面向持续集成的实用流水线感知回归测试优化

Practical Pipeline-Aware Regression Test Optimization for Continuous Integration

Daniel Schwendner, Maximilian Jungwirth, Martin Gruber, Martin Knoche, Daniel Merget, Gordon Fraser

AI总结:

针对大规模多语言单体仓库的持续集成,提出一种基于强化学习的流水线感知回归测试优化方法,利用语言无关特征实现测试优先级排序与选择,并在20周工业数据集上验证有效性。

摘要:

大规模、多语言、单体仓库构成了许多现代复杂软件系统的支柱。为了在保持代码质量一致性的同时支持快速开发周期,持续集成(CI)被广泛应用。然而,在此规模下运行CI不仅成为许多开发者的单点故障,而且需要可能达到可行性极限的计算资源,并导致较长的反馈延迟。为解决这些问题,开发者通常将测试执行拆分为多个流水线,在提交前阶段运行小型快速测试,而在提交后流水线中运行长时间运行和易变(flaky)测试。鉴于许多流水线的运行时间较长,且通过的测试执行占很大比例(在我们的提交前流水线中为98%),因此不仅有必要,而且有潜力通过优先级排序和测试选择来进一步改进。然而,许多先前提出的回归优化技术不适用于工业环境,原因在于:(1)它们依赖复杂且难以获取的特征,如每测试的代码覆盖率,这在大规模多语言环境中不可行;(2)它们不能自动适应快速变化的系统,其中新测试不断被添加或修改;(3)它们并非为区分提交前和提交后流水线的不同目标而设计:提交前测试应优先考虑失败的测试,而提交后流水线应优先考虑通过从通过转为失败或反之来指示非易变变更的测试。为克服这些问题,我们开发了一种轻量级且流水线感知的回归测试优化方法,该方法采用基于语言无关特征训练强化学习模型。我们在一个大型工业数据集上评估了我们的方法,该数据集收集自20周的CI测试执行。在预测...

英文摘要:

Massive, multi-language, monolithic repositories form the backbone of many modern, complex software systems. To ensure consistent code quality while still allowing fast development cycles, Continuous Integration (CI) is commonly applied. However, operating CI at such scale not only leads to a single point of failure for many developers, but also requires computational resources that may reach feasibility limits and cause long feedback latencies. To address these issues, developers commonly split test executions across multiple pipelines, running small and fast tests in pre-submit stages while executing long-running and flaky tests in post-submit pipelines. Given the long runtimes of many pipelines and the substantial proportion of passing test executions (98% in our pre-submit pipelines), there not only a need but also potential for further improvements by prioritizing and selecting tests. However, many previously proposed regression optimization techniques are unfit for an industrial context, because they (1) rely on complex and difficult-to-obtain features like per-test code coverage that are not feasible in large, multi-language environments, (2) do not automatically adapt to rapidly changing systems where new tests are continuously added or modified, and (3) are not designed to distinguish the different objectives of pre- and post-submit pipelines: While pre-submit testing should prioritize failing tests, post-submit pipelines should prioritize tests that indicate non-flaky changes by transitioning from pass to fail outcomes or vice versa. To overcome these issues, we developed a lightweight and pipeline-aware regression test optimization approach that employs Reinforcement Learning models trained on language-agnostic features. We evaluated our approach on a large industry dataset collected over a span of 20 weeks of CI test executions. When predicting...

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑