arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01340cs.CRcs.AIcs.LG
收藏

SubstratumGraphEnv:用于系统攻击路径建模的强化学习环境(RLE)

SubstratumGraphEnv: Reinforcement Learning Environment (RLE) for Modeling System Attack Paths

Bahirah Adewunmi, Edward Raff, Sanjay Purushotham

AI总结:

SubstratumGraphEnv通过图卷积网络建模系统事件序列,自动化识别攻击路径,为网络安全分析提供深度强化学习环境。

摘要:

自动化网络安全分析,特别是潜在攻击路径的识别,面临重大挑战。这主要是由于系统事件的序列性、互联性和进化性,使得大多数人工智能(AI)技术难以有效建模。本文提出了一种强化学习(RL)环境生成框架,模拟在Windows操作系统上执行的过程序列,从而动态建模恶意进程。该方法使用图表示来建模操作系统状态和转换。该图来源于开源的系统监视器(Sysmon)日志。为了解决系统事件类型、字段和日志格式的多样性,开发了一种机制来捕获和建模Sysmon日志中的父子进程。构建了一个Gymnasium环境(SubstratumGraphEnv)以建立RL环境的可感知基础,并构建了一个定制的PyTorch接口(SubstratumBridge)将Gymnasium图转换为深度强化学习(DRL)的观测和离散动作。图卷积网络(GCNs)具体化图的局部和全局状态,这些状态馈入优势演员评论家(A2C)模型的不同策略和评论头。本文的核心贡献在于设计了一个新颖的深度图RL环境,自动化转换序列用户和系统事件,为网络安全分析提供关键上下文。本文为未来研究塑造训练参数和高级奖励塑造提供了基础,同时也提供了关于哪些系统事件属性对训练自主RL代理至关重要的见解。

英文摘要:

Automating network security analysis, particularly the identification of potential attack paths, presents significant challenges. Due in part to the sequential, interconnected, and evolutionary nature of system events which most artificial intelligence (AI) techniques struggle to model effectively. This paper proposes a Reinforcement Learning (RL) environment generation framework that simulates the sequence of processes executed on a Windows operating system, enabling dynamic modeling of malicious processes on a system. This methodology models operating system state and transitions using a graph representation. This graph is derived from open-source System Monitor (Sysmon) logs. To address the variety in system event types, fields, and log formats, a mechanism was developed to capture and model parent-child processes from Sysmon logs. A Gymnasium environment (SubstratumGraphEnv) was constructed to establish the perceptible basis for an RL environment, and a customized PyTorch interface was also built (SubstratumBridge) to translate Gymnasium graphs into Deep Reinforcement Learning (DRL) observations and discrete actions. Graph Convolutional Networks (GCNs) concretize the graph's local and global state, which feed the distinct policy and critic heads of an Advantage Actor-Critic (A2C) model. This work's central contribution lies in the design of a novel deep graphical RL environment that automates translation of sequential user and system events, furnishing crucial context for cybersecurity analysis. This work provides a foundation for future research into shaping training parameters and advanced reward shaping, while also offering insight into which system events attributes are critical to training autonomous RL agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑