让企业安全普及机器学习：一种自维持的攻击检测框架
Democratizing ML for Enterprise Security: A Self-Sustained Attack Detection Framework
AI总结：
本文提出一种自维持的攻击检测框架，通过两阶段混合方法和主动学习，降低企业安全中的机器学习使用门槛，提升威胁检测效率与准确性。
摘要：
尽管在安全领域机器学习取得了进展，但基于规则的检测在安全运营中心仍然普遍存在，因为ML解决方案的资源密集性和技能差距。虽然传统基于规则的方法效率高，但其僵化导致高误报或漏报，并需要持续的手动维护。本文提出了一种新的、两阶段的混合框架，以普及基于机器学习的威胁检测。第一阶段使用故意宽松的YARA规则进行粗粒度过滤，优化为高召回率。第二阶段利用ML分类器过滤掉第一阶段输出中的假阳性。为了克服数据稀缺，系统利用Simula，一个无种子的合成数据生成框架，使安全分析师能够创建高质量的训练数据集，而无需广泛的的数据科学专业知识或预先标记的例子。一个连续的反馈循环将实时调查结果整合进来，以适应性地调整ML模型，防止规则退化。所提出的模型结合主动学习已在一个生产环境中经过严格测试，持续时间长，覆盖数万系统。系统处理初始原始日志量通常达到每天2500亿个事件，通过过滤和ML推断显著减少到每天少量的调查工单。在长时间的实时实验中，由于主动学习特性，模型的精度随时间推移而普遍提高。这种方法提供了一种自维持、低开销、低维护的解决方案，使安全专业人员能够指导模型学习，作为专家“教师”进行指导。
英文摘要：
Despite advancements in machine learning for security, rule-based detection remains prevalent in Security Operations Centers due to the resource intensiveness and skill gap associated with ML solutions. While traditional rule-based methods offer efficiency, their rigidity leads to high false positives or negatives and requires continuous manual maintenance. This paper proposes a novel, two-stage hybrid framework to democratize ML-based threat detection. The first stage employs intentionally loose YARA rules for coarse-grained filtering, optimized for high recall. The second stage utilizes an ML classifier to filter out false positives from the first stage's output. To overcome data scarcity, the system leverages Simula, a seedless synthetic data generation framework, enabling security analysts to create high-quality training datasets without extensive data science expertise or pre-labeled examples. A continuous feedback loop incorporates real-time investigation results to adaptively tune the ML model, preventing rule degradation. This proposed model with active learning has been rigorously tested for a prolonged time in a production environment spanning tens of thousands of systems. The system handles initial raw log volumes often reaching 250 billion events per day, significantly reducing them through filtering and ML inference to a handful of daily tickets for human investigation. Live experiments over an extended timeline demonstrate a general improvement in the model's precision over time due to the active learning feature. This approach offers a self-sustained, low-overhead, and low-maintenance solution, allowing security professionals to guide model learning as expert ``teachers''.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
