大型语言模型（LLM）在网络犯罪论坛中用于网络威胁情报（CTI）的应用
The Use of Large Language Models (LLM) for Cyber Threat Intelligence (CTI) in Cybercrime Forums
AI总结：
本研究评估了基于GPT-3.5-turbo的LLM系统从三个网络犯罪论坛700多个对话中提取CTI信息的性能，平均准确率达96.23%，验证了LLM在网络威胁情报领域的应用价值。
摘要：
大型语言模型（LLMs）可用于分析来自网络犯罪论坛的网络威胁情报（CTI）数据，这些论坛包含大量关于新兴网络威胁的信息和关键讨论。然而，迄今为止，LLMs在此类关键任务中的准确性和效率水平尚未得到彻底评估。因此，本研究评估了一个基于OpenAI GPT-3.5-turbo模型[8]构建的LLM系统在提取CTI信息方面的性能。为此，从三个网络犯罪论坛——XSS、Exploit_in和RAMP——中抽取了超过700个日常对话的随机样本，并指示LLM系统仅通过简单的人类语言指令来总结对话并预测10个关键CTI变量，例如是否有大型组织和/或关键基础设施成为攻击目标。随后，两名编码员审查了每个对话，并评估LLM提取的信息是否准确。该LLM系统表现良好，平均准确率为96.23%，平均精确率为90%，平均召回率为88.2%。研究还发现了多种改进模型的方法，例如需要帮助LLM区分故事与过去事件，以及在提示中注意动词时态的使用。尽管如此，本研究的结果凸显了使用LLMs进行网络威胁情报的相关性。
英文摘要：
Large language models (LLMs) can be used to analyze cyber threat intelligence (CTI) data from cybercrime forums, which contain extensive information and key discussions about emerging cyber threats. However, to date, the level of accuracy and efficiency of LLMs for such critical tasks has yet to be thoroughly evaluated. Hence, this study assesses the performance of an LLM system built on the OpenAI GPT-3.5-turbo model [8] to extract CTI information. To do so, a random sample of more than 700 daily conversations from three cybercrime forums - XSS, Exploit_in, and RAMP - was extracted, and the LLM system was instructed to summarize the conversations and predict 10 key CTI variables, such as whether a large organization and/or a critical infrastructure is being targeted, with only simple human-language instructions. Then, two coders reviewed each conversation and evaluated whether the information extracted by the LLM was accurate. The LLM system performed well, with an average accuracy score of 96.23%, an average precision of 90% and an average recall of 88.2%. Various ways to enhance the model were uncovered, such as the need to help the LLM distinguish between stories and past events, as well as being careful with verb tenses in prompts. Nevertheless, the results of this study highlight the relevance of using LLMs for cyber threat intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
