我的Fuzzer击败他们所有人！模糊器公平性评价与比较框架的建立
My Fuzzer Beats Them All! Developing a Framework for Fair Evaluation and Comparison of Fuzzers
摘要：
模糊技术已经成为识别软件缺陷的最流行的技术之一。为了改善模糊处理，学术文献中最近出现了大量的技术。然而，评估和比较这些技术是具有挑战性的，因为模糊器在生成测试输入时依赖于随机性。通常，现有评价仅部分遵循模糊评价的最佳做法。我们认为，这是双重原因。首先，由于在模糊测试的情况下缺乏全面的经验数据，目前还不清楚拟议的准则是否有必要。第二，目前还没有一个框架，整合统计评估技术，使模糊的公平比较。为了解决这些局限性，我们引入了一种新的模糊评价框架，称为SENF（模糊统计评价）。我们通过利用最广泛的模糊器AFL作为我们的基线模糊器并探索不同评估参数（例如，重复次数或运行时间）、编译器、种子和模糊策略。使用我们的评估框架，我们表明，所谓的小变化的参数可以有一个重大影响的模糊器的测量性能。
英文摘要：
Fuzzing has become one of the most popular techniques to identify bugs in software. To improve the fuzzing process, a plethora of techniques have recently appeared in academic literature. However, evaluating and comparing these techniques is challenging as fuzzers depend on randomness when generating test inputs. Commonly, existing evaluations only partially follow best practices for fuzzing evaluations. We argue that the reason for this are twofold. First, it is unclear if the proposed guidelines are necessary due to the lack of comprehensive empirical data in the case of fuzz testing. Second, there does not yet exist a framework that integrates statistical evaluation techniques to enable fair comparison of fuzzers. To address these limitations, we introduce a novel fuzzing evaluation framework called SENF (Statistical EvaluatioN of Fuzzers). We demonstrate the practical applicability of our framework by utilizing the most wide-spread fuzzer AFL as our baseline fuzzer and exploring the impact of different evaluation parameters (e.g., the number of repetitions or run-time), compilers, seeds, and fuzzing strategies. Using our evaluation framework, we show that supposedly small changes of the parameters can have a major influence on the measured performance of a fuzzer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
