arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2308.04898cs.CRcs.LGcs.SE
收藏

使用大型语言模型分析软件供应链安全故障的实证研究

An Empirical Study on Using Large Language Models to Analyze Software Supply Chain Security Failures

Purdue University(普渡大学)

Tanmay Singla, Dharun Anandayuvaraj, Kelechi G. Kalu, Taylor R. Schorlemmer, James C. Davis

AI总结:

本文实证评估了 GPT 3.5 和 Bard 在复现 CNCF 对 69 起软件供应链安全故障人工分析中的表现,发现 LLMs 在来源充分时能有效分类但尚不能替代人类分析人员。

摘要:

随着我们对软件系统的依赖日益加深,软件供应链中发生泄露事件的后果也变得更加严重。诸如 SolarWinds 和 ShadowHammer 等备受瞩目的网络攻击已造成重大财务和数据损失,凸显了加强网络安全的必要性。预防未来泄露事件的一种方法是研究过去的故障。然而,分析这些故障的传统方法需要人工阅读和总结相关报告。自动化支持可以降低成本,并允许分析更多的故障。自然语言处理(NLP)技术,如大型语言模型(LLMs),可以被用来辅助故障分析。在本研究中,我们评估了大型语言模型(LLMs)分析历史软件供应链泄露事件的能力。我们使用 LLMs 来复现云原生计算基金会(CNCF)成员对 69 起软件供应链安全故障进行的人工分析。我们为 LLMs 开发了提示词,以按四个维度对这些故障进行分类:泄露类型、意图、性质和影响。GPT 3.5 的分类在这些维度上的平均准确率为 68%,Bard 的准确率为 58%。我们报告称,当来源文章足够详细、能够使人工分析人员达成共识时,LLMs 能够有效地刻画软件供应链故障的特征,但尚不能取代人类分析人员。未来的工作可以在这一背景下提高 LLM 的性能,并研究更广泛的文章和故障范围。

英文摘要:

As we increasingly depend on software systems, the consequences of breaches in the software supply chain become more severe. High-profile cyber attacks like those on SolarWinds and ShadowHammer have resulted in significant financial and data losses, underlining the need for stronger cybersecurity. One way to prevent future breaches is by studying past failures. However, traditional methods of analyzing these failures require manually reading and summarizing reports about them. Automated support could reduce costs and allow analysis of more failures. Natural Language Processing (NLP) techniques such as Large Language Models (LLMs) could be leveraged to assist the analysis of failures. In this study, we assessed the ability of Large Language Models (LLMs) to analyze historical software supply chain breaches. We used LLMs to replicate the manual analysis of 69 software supply chain security failures performed by members of the Cloud Native Computing Foundation (CNCF). We developed prompts for LLMs to categorize these by four dimensions: type of compromise, intent, nature, and impact. GPT 3.5s categorizations had an average accuracy of 68% and Bard had an accuracy of 58% over these dimensions. We report that LLMs effectively characterize software supply chain failures when the source articles are detailed enough for consensus among manual analysts, but cannot yet replace human analysts. Future work can improve LLM performance in this context, and study a broader range of articles and failures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑