arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.07235cs.SE
收藏

基于深度学习的漏洞检测:我们到了吗?

Deep Learning based Vulnerability Detection: Are We There Yet?

Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, Baishakhi Ray

摘要:

软件漏洞的自动检测是软件安全中的一个基本问题。现有的程序分析技术遭受高的假阳性或假阴性。深度学习(DL)的最新进展导致人们对将DL应用于自动漏洞检测的兴趣激增。最近的几项研究表明,在检测漏洞方面,准确率高达95%。在本文中,我们问,“如何以及国家的最先进的DL为基础的技术在现实世界中的脆弱性预测的情况下执行?".令我们惊讶的是,我们发现他们的表现下降了50%以上。对导致这种急剧性能下降的原因的系统调查揭示了现有的基于DL的脆弱性预测方法受到训练数据的挑战(例如,数据重复,脆弱类别的分布不切实际等)并且利用模型选择(例如,简单的基于令牌的模型)。因此,这些方法通常不会学习与漏洞的实际原因相关的特征。相反,它们从数据集中学习不相关的工件(例如,具体的变量/函数名称等)。利用这些实证研究结果,我们展示了一个更有原则的方法,数据汇集和模型设计,基于现实的设置脆弱性预测,可以导致更好的解决方案。由此产生的工具的性能显着优于研究的基线:与文献中表现最好的模型相比,准确率提高了33.57%,召回率提高了128.38%。总体而言,本文阐述了现有的基于DL的漏洞预测系统的潜在问题,并绘制了未来的基于DL的漏洞预测研究的路线图。本着这种精神,我们提供了所有支持我们结果的工件:https://git.io/Jf6IA。

英文摘要:

Automated detection of software vulnerabilities is a fundamental problem in software security. Existing program analysis techniques either suffer from high false positives or false negatives. Recent progress in Deep Learning (DL) has resulted in a surge of interest in applying DL for automated vulnerability detection. Several recent studies have demonstrated promising results achieving an accuracy of up to 95% at detecting vulnerabilities. In this paper, we ask, "how well do the state-of-the-art DL-based techniques perform in a real-world vulnerability prediction scenario?". To our surprise, we find that their performance drops by more than 50%. A systematic investigation of what causes such precipitous performance drop reveals that existing DL-based vulnerability prediction approaches suffer from challenges with the training data (e.g., data duplication, unrealistic distribution of vulnerable classes, etc.) and with the model choices (e.g., simple token-based models). As a result, these approaches often do not learn features related to the actual cause of the vulnerabilities. Instead, they learn unrelated artifacts from the dataset (e.g., specific variable/function names, etc.). Leveraging these empirical findings, we demonstrate how a more principled approach to data collection and model design, based on realistic settings of vulnerability prediction, can lead to better solutions. The resulting tools perform significantly better than the studied baseline: up to 33.57% boost in precision and 128.38% boost in recall compared to the best performing model in the literature. Overall, this paper elucidates existing DL-based vulnerability prediction systems' potential issues and draws a roadmap for future DL-based vulnerability prediction research. In that spirit, we make available all the artifacts supporting our results: https://git.io/Jf6IA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑