评《重新审视用于模糊测试的神经程序平滑》
Comment on Revisiting Neural Program Smoothing for Fuzzing
AI总结：
本文指出MLFuzz因实现缺陷、错误评估设置及含噪训练数据而错误否定NEUZZ，修正后证明NEUZZ在FuzzBench上持续优于AFL，并提出科学模糊测试评估建议。
摘要：
MLFuzz是一篇被ACM FSE 2023接收的工作，重新审视了基于机器学习的模糊测试器NEUZZ的性能。我们证明，由于实现中的若干致命缺陷和错误的评估设置，包括持久化模式中的初始化缺陷、程序崩溃、训练数据集收集错误以及模糊测试结果收集错误，其主要结论完全错误。此外，MLFuzz使用了未经充分数据清洗和预处理的含噪训练数据集，这导致NEUZZ的性能急剧下降。我们解决了这些问题，并提供了修正后的实现和评估设置，结果表明在FuzzBench数据集上，NEUZZ始终优于AFL。最后，我们反思了MLFuzz中使用的评估方法，并就公平、科学的模糊测试评估提供了实用建议。
英文摘要：
MLFuzz, a work accepted at ACM FSE 2023, revisits the performance of a machine learning-based fuzzer, NEUZZ. We demonstrate that its main conclusion is entirely wrong due to several fatal bugs in the implementation and wrong evaluation setups, including an initialization bug in persistent mode, a program crash, an error in training dataset collection, and a mistake in fuzzing result collection. Additionally, MLFuzz uses noisy training datasets without sufficient data cleaning and preprocessing, which contributes to a drastic performance drop in NEUZZ. We address these issues and provide a corrected implementation and evaluation setup, showing that NEUZZ consistently performs well over AFL on the FuzzBench dataset. Finally, we reflect on the evaluation methods used in MLFuzz and offer practical advice on fair and scientific fuzzing evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
