Fuzzing has achieved tremendous success in discovering bugs and vulnerabilities in various software systems. Systems under test (SUTs) that take in programming or formal language as inputs, e.g., compilers, runtime engines, constraint solvers, and software libraries with accessible APIs, are especially important as they are fundamental building blocks of software development. However, existing fuzzers for such systems often target a specific language, and thus cannot be easily applied to other languages or even other versions of the same language. Moreover, the inputs generated by existing fuzzers are often limited to specific features of the input language, and thus can hardly reveal bugs related to other or new features. This paper presents Fuzz4All, the first fuzzer that is universal in the sense that it can target many different input languages and many different features of these languages. The key idea behind Fuzz4All is to leverage large language models (LLMs) as an input generation and mutation engine, which enables the approach to produce diverse and realistic inputs for any practically relevant language. To realize this potential, we present a novel autoprompting technique, which creates LLM prompts that are wellsuited for fuzzing, and a novel LLM-powered fuzzing loop, which iteratively updates the prompt to create new fuzzing inputs. We evaluate Fuzz4All on nine systems under test that take in six different languages (C, C++, Go, SMT2, Java and Python) as inputs. The evaluation shows, across all six languages, that universal fuzzing achieves higher coverage than existing, language-specific fuzzers. Furthermore, Fuzz4All has identified 98 bugs in widely used systems, such as GCC, Clang, Z3, CVC5, OpenJDK, and the Qiskit quantum computing platform, with 64 bugs already confirmed by developers as previously unknown.
On the Challenges of Fuzzing Techniques via Large Language Models
School of Electrical and Computer Engineering, The University of Sydney(悉尼大学电气与计算机工程学院);The University of Sydney(悉尼大学);Department of Computer Science, The University of Tokyo(东京大学计算机科学系);The University of Tokyo(东京大学)
FuzzerGym: A Competitive Framework for Fuzzing and Learning
Carnegie Mellon University(卡内基梅隆大学);Georgia Institute of Technology(佐治亚理工学院);Twentieth Century Fox(二十世纪福克斯公司);Starfleet Academy(星际舰队学院);Tyrell Inc.(泰瑞尔公司)
TraceLib: System-Call Bitmap Feedback Mechanism for Language-Agnostic Web Fuzzing
Bernoulli Institute, University of Groningen(格罗宁根大学伯努利学院);Faculty of Engineering, Udayana University(乌达亚纳大学工程学院);Department of Computer Science, University of Cyprus(塞浦路斯大学计算机科学系)
HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
School of Software Technology, Zhejiang University, Ningbo, China(软件技术学院,浙江大学,宁波,中国);The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China(区块链与数据安全国家重点实验室,浙江大学,杭州,中国)
EchoFuzz: Empowering Smart Contract Fuzzing with Large Language Models
Tsinghua University(清华大学);Beihang University(北京航空航天大学);Beijing Normal University(北京师范大学);Zhejiang Gongshang University(浙江工商大学);GoPlus Security(GoPlus安全);University of Oxford(牛津大学);East China Normal University(华东师范大学)
Attention Distance: A Novel Metric for Directed Fuzzing with Large Language Models
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University(广东超高清沉浸媒体技术省重点实验室,北京大学深圳研究生院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);China Mobile Internet CO(中国移动互联网公司)
KQFuzz: Knowledge-Guided Fuzzing for Quantum Libraries via Large Language Models
Shanghai Jiao Tong University(上海交通大学);Nanyang Technological University(南洋理工大学);Hong Kong University of Science and Technology(香港科技大学);Zhejiang University of Technology(浙江工业大学);College of Computing and Data Science(计算与数据科学学院);School of Physical and Mathematical Sciences(物理与数学科学学院)
Belobog: Move Language Fuzzing Framework For Real-World Smart Contracts
Nanyang Technological University(南洋理工大学);Beihang University(北京航空航天大学);Bitslab;Movebit;Hong Kong Polytechnic University(香港理工大学);The Chinese University of Hong Kong(香港中文大学)
SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces
School of Computer Science and Informatics, University of Liverpool(利物浦大学计算机科学与信息学学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)