六种静态分析工具的关键比较:检测、一致性和精度
A Critical Comparison on Six Static Analysis Tools: Detection, Agreement, and Precision
摘要:
背景开发人员使用自动化静态分析工具(ASAT)来控制源代码中的潜在质量问题,包括缺陷和技术债务。工具供应商已经设计了相当多的工具,这使得从业者更难选择最适合他们需要的工具。为了更好地支持开发人员,研究人员一直在对ASAT进行一些研究,以帮助了解其实际能力。目的。尽管到目前为止已经做了很多工作,但仍然缺乏以下方面的知识:(1)哪些源质量问题实际上可以通过静态分析工具警告检测到,(2)它们的一致性如何,以及(3)它们的建议的精度如何。我们的目标是弥合这一差距,提出了一个大规模的比较六个流行的静态分析工具的Java项目:更好的代码中心,检查风格,覆盖扫描,Findbugs,PMD和SonarQube。法我们分析了47个Java项目,并得出了6个国家的实践ASAT提出的警告分类。为了评估他们的一致性,我们通过手动分析(在生产线层面)来比较他们是否发现了相同的问题。最后,我们手动评估工具的精度。结果主要结果报告了反卫星武器警告的全面分类,显示工具之间几乎没有一致性,精确度很低。结论.我们提供了一个分类法,可以是有用的研究人员,从业者和工具供应商映射的工具的当前功能。此外,我们的研究提供了不同工具之间的协议的第一个概述,以及其精度的广泛分析。
英文摘要:
Background. Developers use Automated Static Analysis Tools (ASATs) to control for potential quality issues in source code, including defects and technical debt. Tool vendors have devised quite a number of tools, which makes it harder for practitioners to select the most suitable one for their needs. To better support developers, researchers have been conducting several studies on ASATs to favor the understanding of their actual capabilities. Aims. Despite the work done so far, there is still a lack of knowledge regarding (1) which source quality problems can actually be detected by static analysis tool warnings, (2) what is their agreement, and (3) what is the precision of their recommendations. We aim at bridging this gap by proposing a large-scale comparison of six popular static analysis tools for Java projects: Better Code Hub, CheckStyle, Coverity Scan, Findbugs, PMD, and SonarQube. Method. We analyze 47 Java projects and derive a taxonomy of warnings raised by 6 state-of-the-practice ASATs. To assess their agreement, we compared them by manually analyzing - at line-level - whether they identify the same issues. Finally, we manually evaluate the precision of the tools. Results. The key results report a comprehensive taxonomy of ASATs warnings, show little to no agreement among the tools and a low degree of precision. Conclusions. We provide a taxonomy that can be useful to researchers, practitioners, and tool vendors to map the current capabilities of the tools. Furthermore, our study provides the first overview on the agreement among different tools as well as an extensive analysis of their precision.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。