arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.06537cs.SE
收藏

经典模糊测试的相关性:我们解决了这个问题吗?

The Relevance of Classic Fuzz Testing: Have We Solved This One?

Barton P. Miller, Mengxiao Zhang, Elisa R. Heymann

摘要:

模糊测试已经走过了它的30周年,面对模糊测试技术和工具令人难以置信的进步,问题出现了,如果经典的,基本的模糊技术仍然是有用的和适用的?在这个传统中,我们更新了基本的模糊工具和测试脚本,并将它们应用于Linux,FreeBSD和MacOS上的大量Unix实用程序。和以前一样,我们的失败标准是程序是否崩溃或挂起。我们发现Linux上的74个实用程序中有9个崩溃或挂起,FreeBSD上的78个实用程序中有15个,MacOS上的76个实用程序中有12个。三个平台上共有24个不同的公用事业失败。我们注意到,这些失败率比我们在1995年、2000年和2006年对命令行实用程序可靠性的研究要高一些。在基本的模糊传统中,我们调试每个失败的实用程序,并对失败的原因进行分类。经典的失败类别,如指针和数组错误以及不检查返回码,仍然广泛存在于当前的结果中。此外,我们发现出现了几种新的故障类别。我们提出了这些失败的例子来说明允许它们发生的编程实践。顺便说一下,我们测试了现代程序设计语言(Rust)中有限的实用程序,发现它们的可靠性并不比标准的好。

英文摘要:

As fuzz testing has passed its 30th anniversary, and in the face of the incredible progress in fuzz testing techniques and tools, the question arises if the classic, basic fuzz technique is still useful and applicable? In that tradition, we have updated the basic fuzz tools and testing scripts and applied them to a large collection of Unix utilities on Linux, FreeBSD, and MacOS. As before, our failure criteria was whether the program crashed or hung. We found that 9 crash or hang out of 74 utilities on Linux, 15 out of 78 utilities on FreeBSD, and 12 out of 76 utilities on MacOS. A total of 24 different utilities failed across the three platforms. We note that these failure rates are somewhat higher than our in previous 1995, 2000, and 2006 studies of the reliability of command line utilities. In the basic fuzz tradition, we debugged each failed utility and categorized the causes the failures. Classic categories of failures, such as pointer and array errors and not checking return codes, were still broadly present in the current results. In addition, we found a couple of new categories of failures appearing. We present examples of these failures to illustrate the programming practices that allowed them to happen. As a side note, we tested the limited number of utilities available in a modern programming language (Rust) and found them to be of no better reliability than the standard ones.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑