使用LLM合成的输入生成器进行低成本且全面的非文本输入模糊测试
Low-Cost and Comprehensive Non-textual Input Fuzzing with LLM-Synthesized Input Generators
AI总结:
本文提出G2FUZZ方法,利用LLM合成并变异Python输入生成器生成非文本数据,再结合传统模糊测试工具进一步变异,实现低成本且全面的非文本输入模糊测试,在代码覆盖率和漏洞发现上优于SOTA工具。
摘要:
现代软件通常接受具有高度复杂语法的输入。大型语言模型(LLMs)的最新进展表明,它们可用于合成符合给定输入格式语法的高质量自然语言文本和代码。然而,LLMs通常无法或成本过高,难以生成图像、视频和PDF文件等非文本输出。这一限制阻碍了LLMs在语法感知模糊测试中的应用。我们提出了一种在非文本输入上实现语法感知模糊测试的新方法。我们利用LLMs合成并变异输入生成器(以Python脚本形式),生成符合给定输入格式语法的数据。然后,输入生成器产生的非文本数据由传统模糊测试工具(AFL++)进一步变异,以有效探索软件输入空间。我们的方法名为G2FUZZ,采用混合策略,结合了LLM驱动的全局搜索和工业级模糊测试工具驱动的局部搜索。两个关键优势是:(1)LLMs擅长合成和变异输入生成器,能够跳出局部最优,从而与基于变异的模糊测试工具结合时实现协同效应;(2)除非确实需要,否则较少调用LLMs,从而显著降低LLM使用成本。我们在多种输入格式上评估了G2FUZZ,包括TIFF图像、MP4音频和PDF文件。结果表明,在UNIFUZZ、FuzzBench和MAGMA三个平台上测试的大多数程序中,G2FUZZ在代码覆盖率和漏洞发现方面优于AFL++、Fuzztruction和FormatFuzzer等SOTA工具。
英文摘要:
Modern software often accepts inputs with highly complex grammars. Recent advances in large language models (LLMs) have shown that they can be used to synthesize high-quality natural language text and code that conforms to the grammar of a given input format. Nevertheless, LLMs are often incapable or too costly to generate non-textual outputs, such as images, videos, and PDF files. This limitation hinders the application of LLMs in grammar-aware fuzzing. We present a novel approach to enabling grammar-aware fuzzing over non-textual inputs. We employ LLMs to synthesize and also mutate input generators, in the form of Python scripts, that generate data conforming to the grammar of a given input format. Then, non-textual data yielded by the input generators are further mutated by traditional fuzzers (AFL++) to explore the software input space effectively. Our approach, namely G2FUZZ, features a hybrid strategy that combines a holistic search driven by LLMs and a local search driven by industrial quality fuzzers. Two key advantages are: (1) LLMs are good at synthesizing and mutating input generators and enabling jumping out of local optima, thus achieving a synergistic effect when combined with mutation-based fuzzers; (2) LLMs are less frequently invoked unless really needed, thus significantly reducing the cost of LLM usage. We have evaluated G2FUZZ on a variety of input formats, including TIFF images, MP4 audios, and PDF files. The results show that G2FUZZ outperforms SOTA tools such as AFL++, Fuzztruction, and FormatFuzzer in terms of code coverage and bug finding across most programs tested on three platforms: UNIFUZZ, FuzzBench, and MAGMA.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。