结合探索性分析与自动化分析实现实时数据流中的异常检测
Combining Exploratory Analysis and Automated Analysis for Anomaly Detection in Real-Time Data Streams
AI总结：
本文通过开发实时监控BGP流量的原型系统，探讨探索性分析与自动化分析的互补性，以检测异常并可视化警报上下文，为安全监控系统开发提供见解。
AI中文摘要：
安全分析师在监控对其防御网络至关重要的实时安全信息时，可能会感到不堪重负。他们还倾向于只关注警报的有限部分，因此有可能错过重要事件及其之间的联系。问题的核心在于分析师用来检测、探索和响应网络攻击的系统。安全分析系统的开发者面临的挑战是，开发一个能够以多个抽象级别呈现不同信息来源，同时又要易于使用的系统。在本文中，我们通过测试一个监控实时边界网关协议（BGP）流量以检测可能表明安全威胁的异常的系统开发，来考察探索性分析与自动化分析的互补性。BGP是支持互联网基础设施的关键组成部分；然而，它也高度脆弱，可能被攻击者劫持以传播垃圾邮件或发起拒绝服务攻击。针对BGP基础设施的一些攻击场景可能相当复杂，完全自动化检测此类攻击即使不是不可能，也是困难的。本文做出两项贡献：i）描述了一个用于实时计算指标和威胁警报以及可视化警报上下文的原型平台，ii）讨论了探索性分析（可视化）与自动化分析的交互。本文与对实时安全监控系统的开发或使用感兴趣的学生、安全研究人员和开发者相关。他们将通过实时流式系统的开发，深入了解自动化分析与探索性分析的互补方面。
英文摘要：
Security analysts can become overwhelmed with monitoring real-time security information that is important to help them defend their network. They also tend to focus on a limited portion of the alerts, and therefore risk missing important events and links between them. At the heart of the problem is the system that analysts use to detect, explore, and respond to cyber-attacks. Developers of security analysis systems face the challenge of developing a system that can present different sources of information at multiple levels of abstraction, while also creating a system that is intuitive to use. In this article, we examine the complementary nature of exploratory analysis and automated analysis by testing the development of a system that monitors real-time Border Gateway Protocol (BGP) traffic for anomalies that might indicate security threats. BGP is an essential component for supporting the infrastructure of the Internet; however, it is also highly vulnerable and can be hijacked by attackers to propagate spam or launch denial-of-service attacks. Some of the attack scenarios on the BGP infrastructure can be quite elaborate, and it is difficult, if not impossible, to fully automate the detection of such attacks. This article makes two contributions: i) it describes a prototype platform for computing indicators and threat alerts in real time and for visualizing the context of an alert, and ii) it discusses the interaction of exploratory analysis (visualization) and automated analysis. This article is relevant to students, security researchers, and developers who are interested in the development or use of real-time security monitoring systems. They will gain insights into the complementary aspects of automated analysis and exploratory analysis through the development of a real-time streaming system.
