arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.03064cs.SE
收藏

学习、理解和支持Docker的DevOps工件

Learning from, Understanding, and Supporting DevOps Artifacts for Docker

Jordan Henkel, Christian Bird, Shuvendu K. Lahiri, Thomas Reps

摘要:

随着DevOps工具和框架的使用越来越多,对支持代码以外的工具和技术的需求也越来越大。目前最先进的静态开发人员辅助工具,如Docker,仅限于浅层语法验证。我们确定了在学习、理解和支持开发人员编写DevOps工件领域的三个核心挑战:(i)DevOps工件中的嵌套语言,(ii)规则挖掘,以及(iii)缺乏基于语义规则的分析。为了应对这些挑战,我们引入了一个工具集binnacle,它使我们能够摄取90万个GitHub存储库。围绕Docker,我们提取了大约178,000个独特的Dockerfiles,并确定了由Docker专家编写的Gold Set of Dockerfiles。我们通过一种我们称之为分阶段解析的技术,将我们的AST中有效不可解释的节点数量减少了80%以上,从而解决了挑战(i)。为了解决挑战(ii),我们引入了一种新的规则挖掘技术,能够在我们策划的基准测试中恢复三分之二的规则。通过这种自动化挖掘,我们能够恢复16个在手动规则收集过程中没有找到的新规则。为了解决挑战(iii),我们从Gold Set中的文件提交中手动收集了一组Dockerfiles的规则。这些规则封装了最佳实践,避免了Docker构建失败,并改善了镜像大小和构建延迟。我们创建了一个使用这些规则的分析器,发现GitHub上的Dockerfiles平均违反规则的频率是我们Gold Set中Dockerfiles的五倍。我们还发现,工业Dockerfiles的表现并不比来自GitHub的好。binnacle中的学习规则和分析器可用于在IDE中帮助开发人员创建Dockerfiles,并以事后方式识别现有Dockerfiles中的问题并改进现有Dockerfiles。

英文摘要:

With the growing use of DevOps tools and frameworks, there is an increased need for tools and techniques that support more than code. The current state-of-the-art in static developer assistance for tools like Docker is limited to shallow syntactic validation. We identify three core challenges in the realm of learning from, understanding, and supporting developers writing DevOps artifacts: (i) nested languages in DevOps artifacts, (ii) rule mining, and (iii) the lack of semantic rule-based analysis. To address these challenges we introduce a toolset, binnacle, that enabled us to ingest 900,000 GitHub repositories. Focusing on Docker, we extracted approximately 178,000 unique Dockerfiles, and also identified a Gold Set of Dockerfiles written by Docker experts. We addressed challenge (i) by reducing the number of effectively uninterpretable nodes in our ASTs by over 80% via a technique we call phased parsing. To address challenge (ii), we introduced a novel rule-mining technique capable of recovering two-thirds of the rules in a benchmark we curated. Through this automated mining, we were able to recover 16 new rules that were not found during manual rule collection. To address challenge (iii), we manually collected a set of rules for Dockerfiles from commits to the files in the Gold Set. These rules encapsulate best practices, avoid docker build failures, and improve image size and build latency. We created an analyzer that used these rules, and found that, on average, Dockerfiles on GitHub violated the rules five times more frequently than the Dockerfiles in our Gold Set. We also found that industrial Dockerfiles fared no better than those sourced from GitHub. The learned rules and analyzer in binnacle can be used to aid developers in the IDE when creating Dockerfiles, and in a post-hoc fashion to identify issues in, and to improve, existing Dockerfiles.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑