推进TTP分析：利用大语言模型与检索增强生成的力量
Advancing TTP Analysis: Harnessing the Power of Large Language Models with Retrieval Augmented Generation
AI总结：
本研究比较了编码器LLM的监督微调与解码器LLM的检索增强生成在理解TTPs上的效果，发现RAG方法性能更优，且通用提示更有效。
摘要：
战术、技术和程序（TTPs）概述了攻击者利用漏洞的方法。由于假定专业知识和复杂依赖关系，网络安全从业者解读MITRE ATT&CK框架中的TTPs可能具有挑战性。与此同时，大语言模型（LLMs）的进步引发了探索其在网络安全运营中应用的研究热潮。然而，目前尚不清楚如何高效且恰当地使用LLMs为网络安全等关键领域提供准确响应。这促使我们研究如何更好地利用两类LLMs：小型编码器专用模型（如RoBERTa）和大型解码器专用模型（如GPT-3.5），以理解并总结网络攻击程序预期目的（即战术）的TTPs。本研究比较了编码器专用LLMs的监督微调（SFT）与解码器专用LLMs（无需微调）的检索增强生成（RAG）的使用。SFT和RAG技术均假定通过为每个网络攻击程序提供相关上下文来增强LLMs。我们的研究表明，采用RAG的解码器专用LLMs在性能上优于采用SFT的编码器专用模型，尤其是在RAG提取直接相关上下文的情况下。解码器专用模型的结果可能在实现高“召回率”的同时遭受低“精确率”。我们的发现进一步突出了一个反直觉的观察：更通用的提示往往比更专门定制的提示更能产生更好的网络攻击战术预测。
英文摘要：
Tactics, Techniques, and Procedures (TTPs) outline the methods attackers use to exploit vulnerabilities. The interpretation of TTPs in the MITRE ATT&CK framework can be challenging for cybersecurity practitioners due to presumed expertise and complex dependencies. Meanwhile, advancements with Large Language Models (LLMs) have led to recent surge in studies exploring its uses in cybersecurity operations. It is, however, unclear how LLMs can be used in an efficient and proper way to provide accurate responses for critical domains such as cybersecurity. This leads us to investigate how to better use two types of LLMs: small-scale encoder-only (e.g., RoBERTa) and larger decoder-only (e.g., GPT-3.5) LLMs to comprehend and summarize TTPs with the intended purposes (i.e., tactics) of a cyberattack procedure. This work studies and compares the uses of supervised fine-tuning (SFT) of encoder-only LLMs vs. Retrieval Augmented Generation (RAG) for decoder-only LLMs (without fine-tuning). Both SFT and RAG techniques presumably enhance the LLMs with relevant contexts for each cyberattack procedure. Our studies show decoder-only LLMs with RAG achieves better performance than encoder-only models with SFT, particularly when directly relevant context is extracted by RAG. The decoder-only results could suffer low `Precision' while achieving high `Recall'. Our findings further highlight a counter-intuitive observation that more generic prompts tend to yield better predictions of cyberattack tactics than those that are more specifically tailored.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
