面向分布式数据库的基于多变量日志的异常检测
Multivariate Log-based Anomaly Detection for Distributed Database
AI总结：
针对分布式数据库缺乏多节点、多异常日志数据集及单节点检测误报率高的问题，本文发布首个开源多变量日志数据集，并提出 MultiLog 方法，性能较现有最先进方法提升约 12%。
摘要：
分布式数据库是当今云系统等大规模软件系统的基础基础设施。检测分布式数据库中的异常对于维护软件可用性至关重要。现有方法主要利用 Loghub（一个包含来自各种系统的日志数据集的综合性集合）开发，缺乏专门为分布式数据库定制的数据集，而分布式数据库会表现出独特的异常。此外，还明显缺乏包含多异常、多节点日志的数据集。因此，基于这些数据集构建的模型主要为单机系统设计，无法适用于分布式数据库；而且普遍采用的根据单个节点的异常就判定整个集群异常的方法会导致较高的误报率。本文解决分布式数据库中日志的独特异常和多变量特性问题。我们公开了首个开源的、包含来自分布式数据库的多变量日志的综合性数据集。利用该数据集，我们开展了一项广泛研究，以识别多种数据库异常，并评估利用多变量日志数据进行最先进异常检测的有效性。我们的研究结果表明，仅依赖单个节点的日志不足以在分布式数据库上进行准确的异常检测。利用这些洞察，我们提出了 MultiLog，这是一种为分布式数据库定制的、创新的基于多变量日志的异常检测方法。基于这一新数据集的实验证明了 MultiLog 的优越性，其性能比现有最先进方法高出约 12%。
英文摘要：
Distributed databases are fundamental infrastructures of today's large-scale software systems such as cloud systems. Detecting anomalies in distributed databases is essential for maintaining software availability. Existing approaches, predominantly developed using Loghub-a comprehensive collection of log datasets from various systems-lack datasets specifically tailored to distributed databases, which exhibit unique anomalies. Additionally, there's a notable absence of datasets encompassing multi-anomaly, multi-node logs. Consequently, models built upon these datasets, primarily designed for standalone systems, are inadequate for distributed databases, and the prevalent method of deeming an entire cluster anomalous based on irregularities in a single node leads to a high false-positive rate. This paper addresses the unique anomalies and multivariate nature of logs in distributed databases. We expose the first open-sourced, comprehensive dataset with multivariate logs from distributed databases. Utilizing this dataset, we conduct an extensive study to identify multiple database anomalies and to assess the effectiveness of state-of-the-art anomaly detection using multivariate log data. Our findings reveal that relying solely on logs from a single node is insufficient for accurate anomaly detection on distributed database. Leveraging these insights, we propose MultiLog, an innovative multivariate log-based anomaly detection approach tailored for distributed databases. Our experiments, based on this novel dataset, demonstrate MultiLog's superiority, outperforming existing state-of-the-art methods by approximately 12%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
