arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.07528cs.CRcs.AIcs.CLcs.LG
收藏

SecEncoder:在安全领域日志是你所需的一切

SecEncoder: Logs are All You Need in Security

Microsoft Security AI Research(微软安全人工智能研究部门)

Muhammed Fatih Bulut, Yingqi Liu, Naveed Ahmad, Maximilian Turner, Sami Ait Ouahmane, Cameron Andrews, Lloyd Greenwald

AI总结:

本文提出专门使用安全日志预训练的小型语言模型SecEncoder,通过聚焦安全日志的独特语言和模式克服通用模型领域局限,在多种安全任务上优于BERTlarge等模型,显著提升了安全领域性能。

摘要:

大型和小型语言模型(LMs)通常使用大量文本进行预训练,这些文本来源于维基百科、Book Corpus或通过网络抓取等公开平台。由于接触了广泛的语言数据,这些模型展现出令人印象深刻的泛化能力,并能同时执行多种任务。然而,由于训练数据过于宽泛,它们在特定领域任务上往往表现不佳。本文介绍了SecEncoder,这是一种专门使用安全日志进行预训练的小型语言模型。SecEncoder旨在通过关注安全日志中独特的语言和模式,解决通用LMs在特定领域的局限性。实验结果表明,SecEncoder在各种任务中优于其他主要在自然语言上进行预训练的LMs,如BERTlarge、DeBERTa-v3-large和OpenAI的Embedding(textembedding-ada-002)模型。此外,尽管SecEncoder主要在日志数据上进行预训练,但它在日志分析之外的一系列任务(如事件优先级排序和威胁情报文档检索)上也优于在自然语言上预训练的模型。这表明使用日志进行特定领域的预训练可以显著提升LMs在安全领域的性能。这些发现为未来研究特定于安全的LMs及其潜在应用铺平了道路。

英文摘要:

Large and Small Language Models (LMs) are typically pretrained using extensive volumes of text, which are sourced from publicly accessible platforms such as Wikipedia, Book Corpus, or through web scraping. These models, due to their exposure to a wide range of language data, exhibit impressive generalization capabilities and can perform a multitude of tasks simultaneously. However, they often fall short when it comes to domain-specific tasks due to their broad training data. This paper introduces SecEncoder, a specialized small language model that is pretrained using security logs. SecEncoder is designed to address the domain-specific limitations of general LMs by focusing on the unique language and patterns found in security logs. Experimental results indicate that SecEncoder outperforms other LMs, such as BERTlarge, DeBERTa-v3-large and OpenAI's Embedding (textembedding-ada-002) models, which are pretrained mainly on natural language, across various tasks. Furthermore, although SecEncoder is primarily pretrained on log data, it outperforms models pretrained on natural language for a range of tasks beyond log analysis, such as incident prioritization and threat intelligence document retrieval. This suggests that domain specific pretraining with logs can significantly enhance the performance of LMs in security. These findings pave the way for future research into security-specific LMs and their potential applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑