arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.07927cs.NIcs.LG
收藏

云服务智能监控框架:一种数据驱动方法

Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach

Microsoft(微软公司)

Pooja Srinivas, Fiza Husain, Anjaly Parayil, Ayush Choure, Chetan Bansal, Saravan Rajmohan

AI总结:

提出数据驱动智能监控框架,基于服务属性推荐监控,经微软实证与用户研究验证有效。

摘要:

云服务所有者需要持续监控其服务,以确保高可用性和可靠性。监控中的缺口可能导致事件检测延迟,并对客户产生重大负面影响。当前监控创建过程是临时且反应式的。开发者基于其经验知识,主要采用试错法来创建监控。因此,监控往往覆盖不完整,导致生产问题,或存在冗余,造成噪音和浪费精力。\n在本工作中,我们通过提出一种智能监控框架来解决此问题,该框架基于服务属性推荐监控。我们首先挖掘微软791个生产服务中30,000多个监控的属性,并推导出监控的结构化本体。我们关注两个关键维度:监控什么(资源)以及监控哪些指标。我们进行了广泛的实证研究,并得出了关于微软云服务采用的主要监控类别、其相关维度以及服务属性与本体之间相互关系的关键见解。利用这些见解,我们提出了一种基于深度学习的框架,根据服务属性推荐监控。最后,我们与微软工程师进行了用户研究,证明了所提框架的有效性。所提框架结合本体驱动的投影,成功为大多数资源类别生成了生产质量的推荐。研究中的用户也验证了这一点,他们将该框架的有用性评为4.27分(满分5分)。

英文摘要:

Cloud service owners need to continuously monitor their services to ensure high availability and reliability. Gaps in monitoring can lead to delay in incident detection and significant negative customer impact. Current process of monitor creation is ad-hoc and reactive in nature. Developers create monitors using their tribal knowledge and, primarily, a trial and error based process. As a result, monitors often have incomplete coverage which leads to production issues, or, redundancy which results in noise and wasted effort. In this work, we address this issue by proposing an intelligent monitoring framework that recommends monitors for cloud services based on their service properties. We start by mining the attributes of 30,000+ monitors from 791 production services at Microsoft and derive a structured ontology for monitors. We focus on two crucial dimensions: what to monitor (resources) and which metrics to monitor. We conduct an extensive empirical study and derive key insights on the major classes of monitors employed by cloud services at Microsoft, their associated dimensions, and the interrelationship between service properties and this ontology. Using these insights, we propose a deep learning based framework that recommends monitors based on the service properties. Finally, we conduct a user study with engineers from Microsoft which demonstrates the usefulness of the proposed framework. The proposed framework along with the ontology driven projections, succeeded in creating production quality recommendations for majority of resource classes. This was also validated by the users from the study who rated the framework's usefulness as 4.27 out of 5.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑