arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.04748cs.SEcs.LG
收藏

Fuzz4All:基于大型语言模型的通用模糊测试

Fuzz4All: Universal Fuzzing with Large Language Models

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Stuttgart(斯图加特大学)

Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, Lingming Zhang

AI总结:

本文提出 Fuzz4All,利用 LLM 自动提示与迭代模糊循环实现跨六种语言、九类系统的通用模糊测试,覆盖率优于专用工具并发现 98 个缺陷。

摘要:

模糊测试在发现各类软件系统中的缺陷和漏洞方面取得了巨大成功。以编程语言或形式语言作为输入的被测系统(SUT),例如编译器、运行时引擎、约束求解器以及具有可访问 API 的软件库,尤为重要,因为它们是软件开发的基础构建块。然而,现有的面向此类系统的模糊测试工具通常针对特定语言,因此难以轻松应用于其他语言,甚至难以应用于同一语言的其他版本。此外,现有模糊测试工具生成的输入往往局限于输入语言的特定特性,因而几乎无法揭示与其他特性或新特性相关的缺陷。本文提出 Fuzz4All,这是第一个具有通用性的模糊测试工具,其通用性体现在它能够面向许多不同的输入语言以及这些语言的许多不同特性。Fuzz4All 背后的关键思想是利用大型语言模型(LLM)作为输入生成和变异引擎,这使该方法能够为任何具有实际相关性的语言生成多样且真实的输入。为实现这一潜力,我们提出了一种新颖的自动提示技术,用于创建非常适合模糊测试的 LLM 提示;同时提出了一种新颖的、由 LLM 驱动的模糊测试循环,通过迭代更新提示来创建新的模糊测试输入。我们在九个被测系统上评估了 Fuzz4All,这些系统接受六种不同语言(C、C++、Go、SMT2、Java 和 Python)作为输入。评估表明,在所有六种语言中,通用模糊测试都比现有的、面向特定语言的模糊测试工具实现了更高的覆盖率。此外,Fuzz4All 已在 GCC、Clang、Z3、CVC5、OpenJDK 和 Qiskit 量子计算平台等广泛使用的系统中识别出 98 个缺陷,其中 64 个缺陷已被开发者确认为此前未知。

英文摘要:

Fuzzing has achieved tremendous success in discovering bugs and vulnerabilities in various software systems. Systems under test (SUTs) that take in programming or formal language as inputs, e.g., compilers, runtime engines, constraint solvers, and software libraries with accessible APIs, are especially important as they are fundamental building blocks of software development. However, existing fuzzers for such systems often target a specific language, and thus cannot be easily applied to other languages or even other versions of the same language. Moreover, the inputs generated by existing fuzzers are often limited to specific features of the input language, and thus can hardly reveal bugs related to other or new features. This paper presents Fuzz4All, the first fuzzer that is universal in the sense that it can target many different input languages and many different features of these languages. The key idea behind Fuzz4All is to leverage large language models (LLMs) as an input generation and mutation engine, which enables the approach to produce diverse and realistic inputs for any practically relevant language. To realize this potential, we present a novel autoprompting technique, which creates LLM prompts that are wellsuited for fuzzing, and a novel LLM-powered fuzzing loop, which iteratively updates the prompt to create new fuzzing inputs. We evaluate Fuzz4All on nine systems under test that take in six different languages (C, C++, Go, SMT2, Java and Python) as inputs. The evaluation shows, across all six languages, that universal fuzzing achieves higher coverage than existing, language-specific fuzzers. Furthermore, Fuzz4All has identified 98 bugs in widely used systems, such as GCC, Clang, Z3, CVC5, OpenJDK, and the Qiskit quantum computing platform, with 64 bugs already confirmed by developers as previously unknown.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑