arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.14293cs.CR
收藏

调查贝叶斯垃圾邮件过滤器在检测LLM修改的垃圾邮件中的有效性

Investigating the Effectiveness of Bayesian Spam Filters in Detecting LLM-modified Spam Mails

Malte Josten, Torben Weis

AI总结:

本研究评估SpamAssassin对GPT-3.5 Turbo修改的垃圾邮件的检测能力,发现其误分类率高达73.7%,而字典替换攻击成功率仅0.4%,突显了LLM修改垃圾邮件的严重威胁及改进过滤器的必要性。

摘要:

垃圾邮件和网络钓鱼仍然是网络安全中的关键威胁,约占安全事件的90%。随着这些攻击变得越来越复杂,对强大防御机制的需求也在加剧。贝叶斯垃圾邮件过滤器,如广泛采用的开源SpamAssassin,是这场斗争中的重要工具。然而,大型语言模型(LLMs)如ChatGPT的出现带来了新的挑战。这些模型不仅功能强大且易于获取,而且使用成本低廉,这引发了人们对它们被滥用于制作逃避传统垃圾邮件过滤器的复杂垃圾邮件的担忧。本研究旨在评估SpamAssassin对LLM修改的电子邮件内容的鲁棒性和有效性。我们开发了一个管道来测试这一漏洞。我们的管道使用GPT-3.5 Turbo修改垃圾邮件,并评估SpamAssassin正确分类这些修改后邮件的能力。结果显示,SpamAssassin将高达73.7%的LLM修改的垃圾邮件误分类为合法邮件。相比之下,一种更简单的字典替换攻击的最大成功率仅为0.4%。这些发现突显了LLM修改的垃圾邮件构成的重大威胁,尤其是考虑到此类攻击的成本效益(每封邮件0.17美分)。本文为当前垃圾邮件过滤器的漏洞以及持续改进网络安全措施的必要性提供了关键见解。

英文摘要:

Spam and phishing remain critical threats in cybersecurity, responsible for nearly 90% of security incidents. As these attacks grow in sophistication, the need for robust defensive mechanisms intensifies. Bayesian spam filters, like the widely adopted open-source SpamAssassin, are essential tools in this fight. However, the emergence of large language models (LLMs) such as ChatGPT presents new challenges. These models are not only powerful and accessible, but also inexpensive to use, raising concerns about their misuse in crafting sophisticated spam emails that evade traditional spam filters. This work aims to evaluate the robustness and effectiveness of SpamAssassin against LLM-modified email content. We developed a pipeline to test this vulnerability. Our pipeline modifies spam emails using GPT-3.5 Turbo and assesses SpamAssassin's ability to classify these modified emails correctly. The results show that SpamAssassin misclassified up to 73.7% of LLM-modified spam emails as legitimate. In contrast, a simpler dictionary-replacement attack showed a maximum success rate of only 0.4%. These findings highlight the significant threat posed by LLM-modified spam, especially given the cost-efficiency of such attacks (0.17 cents per email). This paper provides crucial insights into the vulnerabilities of current spam filters and the need for continuous improvement in cybersecurity measures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑