arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.07232cs.CRcs.AIcs.SE
收藏

ChatGPT模型在漏洞检测中的评估

Evaluation of ChatGPT Model for Vulnerability Detection

OpenAI(开放人工智能公司)

Anton Cheshkov, Pavel Zadorozhny, Rodion Levichev

AI总结:

本文评估了ChatGPT和GPT-3在真实世界CWE漏洞数据集上的二分类与多标签漏洞检测性能,发现ChatGPT并不优于虚拟分类器。

摘要:

在本技术报告中,我们评估了ChatGPT和GPT-3模型在代码漏洞检测任务中的性能。我们的评估基于真实世界数据集,采用CWE漏洞的二分类和多标签分类任务。我们决定评估该模型,是因为它在其他基于代码的任务上表现出色,例如解决编程挑战和高层次理解代码。然而,我们发现ChatGPT模型在代码漏洞检测的二分类和多标签分类任务中,其性能并不优于虚拟分类器。

英文摘要:

In this technical report, we evaluated the performance of the ChatGPT and GPT-3 models for the task of vulnerability detection in code. Our evaluation was conducted on our real-world dataset, using binary and multi-label classification tasks on CWE vulnerabilities. We decided to evaluate the model because it has shown good performance on other code-based tasks, such as solving programming challenges and understanding code at a high level. However, we found that the ChatGPT model performed no better than a dummy classifier for both binary and multi-label classification tasks for code vulnerability detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑