arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.03214cs.CRcs.AIcs.LG
收藏

基于转换器的软件漏洞检测语言模型

Transformer-Based Language Models for Software Vulnerability Detection

CSIRO Data61(澳大利亚联邦科学与工业研究组织数据61研究所);Institute of Computer Science, Polish Academy of Sciences(波兰科学院计算机科学研究所)

Chandra Thapa, Seung Ick Jang, Muhammad Ejaz Ahmed, Seyit Camtepe, Josef Pieprzyk, Surya Nepal

摘要:

大型的基于transformer的语言模型在自然语言处理中表现出优异的性能。通过考虑这些模型在一个领域中获得的知识到其他相关领域的可转移性,以及自然语言与高级程序设计语言(如C/C++)的接近性,本文研究了如何利用基于(大型)transformer的语言模型检测软件漏洞,以及这些模型对漏洞检测任务的影响。在这方面,首先,一个系统的(内聚)框架,详细介绍了源代码翻译,模型准备和推理。然后,使用具有与库函数调用、指针使用、数组使用和算术表达式对应的多个漏洞的C/C++源代码的软件漏洞数据集进行实证分析。我们的实验结果表明,语言模型在漏洞检测的良好性能。此外,这些语言模型具有更好的性能指标,如F1分数,比当代模型,即双向长短期记忆和双向门控递归单元。由于计算资源、平台、库和依赖关系的要求,使用语言模型进行实验总是具有挑战性的。因此,本文还分析了流行的平台,以有效地微调这些模型,并在选择平台时提出建议。

英文摘要:

The large transformer-based language models demonstrate excellent performance in natural language processing. By considering the transferability of the knowledge gained by these models in one domain to other related domains, and the closeness of natural languages to high-level programming languages, such as C/C++, this work studies how to leverage (large) transformer-based language models in detecting software vulnerabilities and how good are these models for vulnerability detection tasks. In this regard, firstly, a systematic (cohesive) framework that details source code translation, model preparation, and inference is presented. Then, an empirical analysis is performed with software vulnerability datasets with C/C++ source codes having multiple vulnerabilities corresponding to the library function call, pointer usage, array usage, and arithmetic expression. Our empirical results demonstrate the good performance of the language models in vulnerability detection. Moreover, these language models have better performance metrics, such as F1-score, than the contemporary models, namely bidirectional long short-term memory and bidirectional gated recurrent unit. Experimenting with the language models is always challenging due to the requirement of computing resources, platforms, libraries, and dependencies. Thus, this paper also analyses the popular platforms to efficiently fine-tune these models and present recommendations while choosing the platforms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑