SAC-AP:基于软Actor Critic的深度强化学习警报优先级划分
SAC-AP: Soft Actor Critic based Deep Reinforcement Learning for Alert Prioritization
摘要:
入侵检测系统(IDS)产生大量的假警报,这使得它很难检查真阳性。因此,警报优先级在决定从IDS生成的大量警报中调查哪些警报方面起着至关重要的作用。最近,基于深度强化学习(DRL)的深度确定性策略梯度(DDPG)离线策略方法已被证明与其他最先进的方法相比,可以实现更好的警报优先级排序结果。然而,DDPG容易出现过拟合问题。此外,它也有一个贫穷的探索能力,因此它不适合与随机环境的问题。为了解决这些局限性,我们提出了一个软演员-评论家为基础的DRL算法的警报优先级(SAC-AP),一个政策外的方法,基于最大熵强化学习框架,旨在最大限度地提高预期的回报,同时也最大限度地提高熵。在此基础上,将攻防双方的博弈模型化为零和博弈,并利用双预言机框架得到了近似混合策略纳什均衡(MSNE). SAC-AP发现强大的警报调查策略,并计算纯策略的最佳反应对手的混合策略。我们提出了整体设计的SAC-AP和评估其性能相比,其他国家的最先进的警报优先级的方法。我们认为辩护方的损失,即,防御者无法调查由于攻击而触发的警报,作为性能指标。我们的研究结果表明,SAC-AP实现了高达30%的防御者的损失相比,DDPG为基础的警报优先级的方法,从而提供了更好的保护,防止入侵。此外,当SAC-AP与其他传统的警报优先级排序方法(包括Uniform、GAIN、RIO和Suricata)相比时,其优势甚至更高。
英文摘要:
Intrusion detection systems (IDS) generate a large number of false alerts which makes it difficult to inspect true positives. Hence, alert prioritization plays a crucial role in deciding which alerts to investigate from an enormous number of alerts that are generated by IDS. Recently, deep reinforcement learning (DRL) based deep deterministic policy gradient (DDPG) off-policy method has shown to achieve better results for alert prioritization as compared to other state-of-the-art methods. However, DDPG is prone to the problem of overfitting. Additionally, it also has a poor exploration capability and hence it is not suitable for problems with a stochastic environment. To address these limitations, we present a soft actor-critic based DRL algorithm for alert prioritization (SAC-AP), an off-policy method, based on the maximum entropy reinforcement learning framework that aims to maximize the expected reward while also maximizing the entropy. Further, the interaction between an adversary and a defender is modeled as a zero-sum game and a double oracle framework is utilized to obtain the approximate mixed strategy Nash equilibrium (MSNE). SAC-AP finds robust alert investigation policies and computes pure strategy best response against opponent's mixed strategy. We present the overall design of SAC-AP and evaluate its performance as compared to other state-of-the art alert prioritization methods. We consider defender's loss, i.e., the defender's inability to investigate the alerts that are triggered due to attacks, as the performance metric. Our results show that SAC-AP achieves up to 30% decrease in defender's loss as compared to the DDPG based alert prioritization method and hence provides better protection against intrusions. Moreover, the benefits are even higher when SAC-AP is compared to other traditional alert prioritization methods including Uniform, GAIN, RIO and Suricata.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。