arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.04146cs.SEcs.AIcs.LG
收藏

代码气味在机器学习项目中的流行

The Prevalence of Code Smells in Machine Learning projects

Delft University of Technology(代尔夫特理工大学);ING(ING集团)

Bart van Oort, Luís Cruz, Maurício Aniche, Arie van Deursen

摘要:

人工智能(AI)和机器学习(ML)在当前的计算机科学领域中无处不在。然而,在这一领域仍然缺乏软件工程经验和最佳实践。静态代码分析是这样一种最佳实践,可以用来发现代码气味,即,源代码中的(潜在)缺陷、重构机会以及违反通用编码标准。我们的研究旨在发现ML项目中最普遍的代码气味。我们收集了74个开源机器学习项目的数据集,安装了它们的依赖项,并在上面运行了Pylint。这导致了每个类别中所有检测到的代码气味的前20个。对这些气味的手动分析主要表明代码重复是普遍存在的,并且由于其与数学符号的相似性,用于标识符命名风格的PEP 8约定可能并不总是适用于ML代码。然而,更有趣的是,我们发现了ML项目的可维护性和可重复性的几个主要障碍,主要与Python项目的依赖管理有关。我们还发现Pylint无法可靠地检查导入依赖项的正确使用,包括PyTorch等突出的ML库。

英文摘要:

Artificial Intelligence (AI) and Machine Learning (ML) are pervasive in the current computer science landscape. Yet, there still exists a lack of software engineering experience and best practices in this field. One such best practice, static code analysis, can be used to find code smells, i.e., (potential) defects in the source code, refactoring opportunities, and violations of common coding standards. Our research set out to discover the most prevalent code smells in ML projects. We gathered a dataset of 74 open-source ML projects, installed their dependencies and ran Pylint on them. This resulted in a top 20 of all detected code smells, per category. Manual analysis of these smells mainly showed that code duplication is widespread and that the PEP8 convention for identifier naming style may not always be applicable to ML code due to its resemblance with mathematical notation. More interestingly, however, we found several major obstructions to the maintainability and reproducibility of ML projects, primarily related to the dependency management of Python projects. We also found that Pylint cannot reliably check for correct usage of imported dependencies, including prominent ML libraries such as PyTorch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑