Previous learning-based vulnerability detection methods relied on either medium-sized pre-trained models or smaller neural networks from scratch. Recent advancements in Large Pre-Trained Language Models (LLMs) have showcased remarkable few-shot learning capabilities in various tasks. However, the effectiveness of LLMs in detecting software vulnerabilities is largely unexplored. This paper aims to bridge this gap by exploring how LLMs perform with various prompts, particularly focusing on two state-of-the-art LLMs: GPT-3.5 and GPT-4. Our experimental results showed that GPT-3.5 achieves competitive performance with the prior state-of-the-art vulnerability detection approach and GPT-4 consistently outperformed the state-of-the-art.
LLM-based Vulnerability Detection at Project Scale: An Empirical Study
School of Computer Software, College of Intelligence and Computing, Tianjin University（天津大学计算机科学与技术学院）；International Joint Institute of Tianjin University（天津大学国际联合学院）；Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, School of Computer Science, Peking University（北京大学计算机科学学院，教育部高可信软件技术重点实验室）
Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection
Singapore Management University（新加坡管理大学）；Hanoi University of Science and Technology（河内理工大学）；The University of Melbourne（墨尔本大学）；University of Melbourne（墨尔本大学）
Vulnerability Detection: From Formal Verification to Large Language Models and Hybrid Approaches: A Comprehensive Overview
Eötvös Loránd University (ELTE)（厄特沃什·罗兰大学（ELTE））；Technology Innovation Institute (TII)（技术创新研究院（TII））；University of Oslo（奥斯陆大学）；Cyentific AS（赛恩蒂菲克公司）；Guelma University（盖勒马大学）；The University of Manchester（曼彻斯特大学）；Federal University of Amazonas（亚马逊联邦大学）
