arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.12199cs.SE
收藏

机器学习项目中CI/CD流水线演化的实证分析

Empirical Analysis on CI/CD Pipeline Evolution in Machine Learning Projects

Dhia Elhaq Rzig, Alaa Houerbi, Rahul Ghanshyam Chavan, Foyzul Hassan

AI总结:

本研究首次针对ML项目的CI/CD配置演化开展实证分析,通过多维度分析识别变更类别、模式及不良实践,发现经验丰富的开发者更倾向于修改CI/CD配置。

摘要:

机器学习(ML)的日益普及以及ML组件与其他软件制品的集成,推动了持续集成与持续交付(CI/CD)工具的应用,例如Travis CI、GitHub Actions等,这些工具能够为ML项目实现更快的集成与测试。这类CI/CD配置与服务需要在项目生命周期内保持同步。已有多项研究探讨了传统软件系统中CI/CD配置与服务在使用过程中的变化情况,但目前关于ML项目中CI/CD配置与服务如何变化的认知仍十分有限。\n为填补这一认知空白,本研究首次针对ML软件系统的CI/CD配置演化开展了实证分析。我们手动分析了从508个开源ML项目中收集的343次提交,识别出ML项目中常见的CI/CD配置变更类别,并构建了包含14种CI/CD与ML组件协同变更的分类体系。此外,我们开发了一款CI/CD配置变更聚类工具,在15634次提交中识别出频繁出现的CI/CD配置变更模式。我们还对修改CI/CD配置的ML开发者的专业水平进行了衡量。基于上述分析,我们发现61.8%的提交包含构建策略的变更,与通用开源项目相比,性能和可维护性相关的变更占比极低。另外,协同演化分析表明,在很多情况下CI/CD配置会因不良实践(如直接引入依赖、未使用标准化测试框架)而发生不必要的变更。通过变更模式分析还发现了更多不良实践,包括使用已弃用的设置以及依赖通用构建语言。最后,我们的开发者专业水平分析显示,经验丰富的开发者更倾向于修改CI/CD配置。

英文摘要:

The growing popularity of machine learning (ML) and the integration of ML components with other software artifacts has led to the use of continuous integration and delivery (CI/CD) tools, such as Travis CI, GitHub Actions, etc. that enable faster integration and testing for ML projects. Such CI/CD configurations and services require synchronization during the life cycle of the projects. Several works discussed how CI/CD configuration and services change during their usage in traditional software systems. However, there is very limited knowledge of how CI/CD configuration and services change in ML projects. To fill this knowledge gap, this work presents the first empirical analysis of how CI/CD configuration evolves for ML software systems. We manually analyzed 343 commits collected from 508 open-source ML projects to identify common CI/CD configuration change categories in ML projects and devised a taxonomy of 14 co-changes in CI/CD and ML components. Moreover, we developed a CI/CD configuration change clustering tool that identified frequent CI/CD configuration change patterns in 15,634 commits. Furthermore, we measured the expertise of ML developers who modify CI/CD configurations. Based on this analysis, we found that 61.8% of commits include a change to the build policy and minimal changes related to performance and maintainability compared to general open-source projects. Additionally, the co-evolution analysis identified that CI/CD configurations, in many cases, changed unnecessarily due to bad practices such as the direct inclusion of dependencies and a lack of usage of standardized testing frameworks. More practices were found through the change patterns analysis consisting of using deprecated settings and reliance on a generic build language. Finally, our developer's expertise analysis suggests that experienced developers are more inclined to modify CI/CD configurations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑