SoK: 深度强化学习在网络安全中的陷阱
SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity
AI总结：
本文系统梳理了深度强化学习应用于网络安全时的11个方法学陷阱，通过分析66篇论文量化其普遍性，并在三个典型场景中实验验证其影响，最后提出改进建议。
摘要：
深度强化学习（DRL）在需要序贯决策的领域取得了显著成功，推动了其在网络安全问题中的应用。然而，将DRL从实验室模拟迁移到定制化的网络环境可能会引入许多问题。大多数网络安全任务通常具有对抗性、非平稳性和部分可观测性，这进一步加剧了问题。在本文中，我们识别并系统化了DRL4Sec文献中在环境建模、智能体训练、性能评估和系统部署阶段频繁出现的11个方法学陷阱。通过分析66篇重要的DRL4Sec论文（2018-2025年），我们量化了每个陷阱的普遍性，发现每篇论文平均有超过五个陷阱。我们通过在（i）自主网络防御、（ii）对抗性恶意软件创建和（iii）Web安全测试环境中的受控实验，展示了这些陷阱的实际影响。最后，我们针对每个陷阱提供了可操作的建议，以支持开发更严谨、更可部署的基于DRL的安全系统。
英文摘要：
Deep Reinforcement Learning (DRL) has achieved remarkable success in domains requiring sequential decision-making, motivating its application to cybersecurity problems. However, transitioning DRL from laboratory simulations to bespoke cyber environments can introduce numerous issues. This is further exacerbated by the often adversarial, non-stationary, and partially-observable nature of most cybersecurity tasks. In this paper, we identify and systematize 11 methodological pitfalls that frequently occur in DRL for cybersecurity (DRL4Sec) literature across the stages of environment modeling, agent training, performance evaluation, and system deployment. By analyzing 66 significant DRL4Sec papers (2018-2025), we quantify the prevalence of each pitfall and find an average of over five pitfalls per paper. We demonstrate the practical impact of these pitfalls using controlled experiments in (i) autonomous cyber defense, (ii) adversarial malware creation, and (iii) web security testing environments. Finally, we provide actionable recommendations for each pitfall to support the development of more rigorous and deployable DRL-based security systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
