SecureBERT：一种面向网络安全的领域特定语言模型
SecureBERT: A Domain-Specific Language Model for Cybersecurity
摘要：
自然语言处理（NLP）最近在网络安全领域获得了广泛的关注，特别是在网络威胁情报（CTI）和网络自动化方面。连接和自动化的增加彻底改变了世界经济和文化基础设施，同时也带来了网络攻击方面的风险。CTI是帮助网络安全分析师做出智能安全决策的信息，通常以自然语言文本的形式提供，必须通过自动程序将其转换为机器可读格式，然后才能用于自动安全措施。本文提出了SecureBERT，一种网络安全语言模型，能够捕获网络安全文本中的文本内涵（例如，CTI），因此成功地自动化了许多关键的网络安全任务，否则这些任务将依赖于人类的专业知识和耗时的手动工作。SecureBERT已经使用大量网络安全文本语料库进行了训练。为了使SecureBERT不仅在保留一般英语理解方面有效，而且在应用于具有网络安全含义的文本时也有效，我们开发了一个定制的标记器以及一种改变预训练权重的方法。SecureBERT使用标准的Masked语言模型（MLM）测试以及两个额外的标准NLP任务进行评估。我们的评估研究表明，SecureBERT \footnote{\url{https://github.com/ehsanaghaei/SecureBERT}}优于现有的类似模型，证实了其解决网络安全中关键NLP任务的能力。
英文摘要：
Natural Language Processing (NLP) has recently gained wide attention in cybersecurity, particularly in Cyber Threat Intelligence (CTI) and cyber automation. Increased connection and automation have revolutionized the world's economic and cultural infrastructures, while they have introduced risks in terms of cyber attacks. CTI is information that helps cybersecurity analysts make intelligent security decisions, that is often delivered in the form of natural language text, which must be transformed to machine readable format through an automated procedure before it can be used for automated security measures. This paper proposes SecureBERT, a cybersecurity language model capable of capturing text connotations in cybersecurity text (e.g., CTI) and therefore successful in automation for many critical cybersecurity tasks that would otherwise rely on human expertise and time-consuming manual efforts. SecureBERT has been trained using a large corpus of cybersecurity text.To make SecureBERT effective not just in retaining general English understanding, but also when applied to text with cybersecurity implications, we developed a customized tokenizer as well as a method to alter pre-trained weights. The SecureBERT is evaluated using the standard Masked Language Model (MLM) test as well as two additional standard NLP tasks. Our evaluation studies show that SecureBERT\footnote{\url{https://github.com/ehsanaghaei/SecureBERT}} outperforms existing similar models, confirming its capability for solving crucial NLP tasks in cybersecurity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
