On the Challenges of Fuzzing Techniques via Large Language Models
School of Electrical and Computer Engineering, The University of Sydney（悉尼大学电气与计算机工程学院）；The University of Sydney（悉尼大学）；Department of Computer Science, The University of Tokyo（东京大学计算机科学系）；The University of Tokyo（东京大学）
In the modern era where software plays a pivotal role, software security and vulnerability analysis are essential for secure software development. Fuzzing test, as an efficient and traditional software testing method, has been widely adopted across various domains. Meanwhile, the rapid development in Large Language Models (LLMs) has facilitated their application in the field of software testing, demonstrating remarkable performance. As existing fuzzing test techniques are not fully automated and software vulnerabilities continue to evolve, there is a growing interest in leveraging large language models to generate fuzzing test. In this paper, we present a systematic overview of the developments that utilize large language models for the fuzzing test. To our best knowledge, this is the first work that covers the intersection of three areas, including LLMs, fuzzing test, and fuzzing test generated based on LLMs. A statistical analysis and discussion of the literature are conducted by summarizing the state-of-the-art methods up to date of the submission. Our work also investigates the potential for widespread deployment and application of fuzzing test techniques generated by LLMs in the future, highlighting their promise for advancing automated software testing practices.
HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
School of Software Technology, Zhejiang University, Ningbo, China（软件技术学院，浙江大学，宁波，中国）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China（区块链与数据安全国家重点实验室，浙江大学，杭州，中国）
Attention Distance: A Novel Metric for Directed Fuzzing with Large Language Models
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University（广东超高清沉浸媒体技术省重点实验室，北京大学深圳研究生院）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；China Mobile Internet CO（中国移动互联网公司）
KQFuzz: Knowledge-Guided Fuzzing for Quantum Libraries via Large Language Models
Shanghai Jiao Tong University（上海交通大学）；Nanyang Technological University（南洋理工大学）；Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University of Technology（浙江工业大学）；College of Computing and Data Science（计算与数据科学学院）；School of Physical and Mathematical Sciences（物理与数学科学学院）
Unified Hallucination Fuzzing for Multimodal Large Language Models
National University of Singapore（新加坡国立大学）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Renmin University of China（中国人民大学）；Zhejiang University（浙江大学）；Hupan Lab（湖畔实验室）；University of California, Berkeley（加州大学伯克利分校）；Rochester Institute of Technology（罗切斯特理工学院）；The Hong Kong University of Science and Technology（香港科技大学）
SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces
School of Computer Science and Informatics, University of Liverpool（利物浦大学计算机科学与信息学学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Testing Storage-System Correctness: Challenges, Fuzzing Limitations, and AI-Augmented Opportunities
Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS)（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Laboratory（中关村实验室）
FuzzerGym: A Competitive Framework for Fuzzing and Learning
Carnegie Mellon University（卡内基梅隆大学）；Georgia Institute of Technology（佐治亚理工学院）；Twentieth Century Fox（二十世纪福克斯公司）；Starfleet Academy（星际舰队学院）；Tyrell Inc.（泰瑞尔公司）
TraceLib: System-Call Bitmap Feedback Mechanism for Language-Agnostic Web Fuzzing
Bernoulli Institute, University of Groningen（格罗宁根大学伯努利学院）；Faculty of Engineering, Udayana University（乌达亚纳大学工程学院）；Department of Computer Science, University of Cyprus（塞浦路斯大学计算机科学系）
EchoFuzz: Empowering Smart Contract Fuzzing with Large Language Models
Tsinghua University（清华大学）；Beihang University（北京航空航天大学）；Beijing Normal University（北京师范大学）；Zhejiang Gongshang University（浙江工商大学）；GoPlus Security（GoPlus安全）；University of Oxford（牛津大学）；East China Normal University（华东师范大学）
