arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.14372cs.SEcs.AIcs.CRcs.LG
收藏

SCoPE:评估大语言模型在软件漏洞检测中的表现

SCoPE: Evaluating LLMs for Software Vulnerability Detection

José Gonçalves, Tiago Dias, Eva Maia, Isabel Praça

AI总结:

本研究提出SCoPE框架优化CVEFixes数据集,并微调三个预训练大语言模型进行软件漏洞检测,最佳模型F1分数达53%。

摘要:

近年来,代码安全变得越来越重要,尤其是在互联技术兴起的背景下。在软件开发过程中及早检测漏洞已显示出诸多益处。因此,科学界开始利用机器学习对源代码漏洞进行自动检测。本研究探索并优化了CVEFixes数据集,该数据集通常用于训练代码相关任务的模型,特别是C/C++子集。为此,提出了源代码处理引擎(SCoPE),这是一个由策略化技术组成的框架,可用于缩减C/C++函数规模并对其进行规范化。SCoPE生成的输出被用于创建新版本的CVEFixes。随后,这个精炼后的数据集被用于特征表示分析,以评估该工具代码处理技术的有效性,包括微调三个预训练的大语言模型用于软件漏洞检测。结果表明,SCoPE成功帮助识别了所评估子集中的905个重复项。大语言模型的结果与文献中关于其适用于软件漏洞检测的结论一致,最佳模型达到了53%的F1分数。

英文摘要:

In recent years, code security has become increasingly important, especially with the rise of interconnected technologies. Detecting vulnerabilities early in the software development process has demonstrated numerous benefits. Consequently, the scientific community started using machine learning for automated detection of source code vulnerabilities. This work explores and refines the CVEFixes dataset, which is commonly used to train models for code-related tasks, specifically the C/C++ subset. To this purpose, the Source Code Processing Engine (SCoPE), a framework composed of strategized techniques that can be used to reduce the size and normalize C/C++ functions is presented. The output generated by SCoPE was used to create a new version of CVEFixes. This refined dataset was then employed in a feature representation analysis to assess the effectiveness of the tool's code processing techniques, consisting of fine-tuning three pre-trained LLMs for software vulnerability detection. The results show that SCoPE successfully helped to identify 905 duplicates within the evaluated subset. The LLM results corroborate with the literature regarding their suitability for software vulnerability detection, with the best model achieving 53% F1-score.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑