arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.11147cs.SEcs.AI
收藏

Vul-RAG:通过知识级RAG增强基于大语言模型的漏洞检测

Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG

Fudan University(复旦大学);Alibaba Group(阿里巴巴集团);Sun Yat-sen University(中山大学)

Xueying Du, Geng Zheng, Kaixin Wang, Yi Zou, Yujia Wang, Wentai Deng, Jiayi Feng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, Yiling Lou

AI总结:

针对大语言模型在区分漏洞代码与相似已修复良性代码时准确率低、难捕捉漏洞根因的问题,提出知识级检索增强生成框架Vul-RAG,利用历史漏洞提炼的多维度知识增强模型,显著提升检测准确率,还可辅助人工检测并发现未知漏洞。

摘要:

尽管大语言模型(LLMs)在漏洞检测领域已展现出可观潜力,但本研究揭示了其在区分存在漏洞的代码与相似但已修复的良性代码时存在局限性——准确率仅为0.06-0.14。研究表明,LLMs在漏洞检测过程中难以捕捉漏洞的根本成因。为应对这一挑战,我们提出利用从历史漏洞及修复方案中提炼的多维度漏洞知识来增强LLMs的性能。我们设计了一种全新的知识级检索增强生成框架Vul-RAG,该框架将LLMs识别漏洞代码与已修复代码的准确率提升了16%-24%。此外,Vul-RAG生成的漏洞知识还可进一步发挥两大作用:(1)作为高质量解释材料,将人工检测的准确率从60%提升至77%;(2)在最新版Linux内核版本中检测出10个此前未知的漏洞,其中6个已分配CVE编号。

英文摘要:

Although LLMs have shown promising potential in vulnerability detection, this study reveals their limitations in distinguishing between vulnerable and similar-but-benign patched code (only 0.06 - 0.14 accuracy). It shows that LLMs struggle to capture the root causes of vulnerabilities during vulnerability detection. To address this challenge, we propose enhancing LLMs with multi-dimensional vulnerability knowledge distilled from historical vulnerabilities and fixes. We design a novel knowledge-level Retrieval-Augmented Generation framework Vul-RAG, which improves LLMs with an accuracy increase of 16% - 24% in identifying vulnerable and patched code. Additionally, vulnerability knowledge generated by Vul-RAG can further (1) serve as high-quality explanations to improve manual detection accuracy (from 60% to 77%), and (2) detect 10 previously-unknown bugs in the recent Linux kernel release with 6 assigned CVEs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑