arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.03665cs.DCcs.AIcs.CLcs.LG
收藏

DeepTriage:云服务中事件的自动传输协助

DeepTriage: Automated Transfer Assistance for Incidents in Cloud Services

Microsoft(微软)

Phuong Pham, Vivek Jain, Lukas Dauterman, Justin Ormont, Navendu Jain

摘要:

随着云服务的增长和产生高收入,这些服务的停机成本变得非常昂贵。为了减少损失和服务停机时间,一个关键的主要步骤是执行事件分类,及时将服务事件分配给正确的负责团队。不正确的分配可能会导致额外的事件重新路由,并使缓解时间增加10倍。然而,大型云服务中的自动化事件分类面临许多挑战:(1)来自大量团队的高度不平衡的事件分布,(2)输入数据或数据源的格式多种多样,(3)扩展以满足生产级要求,以及(4)获得工程师对使用机器学习建议的信任。为了应对这些挑战,我们引入了DeepTriage,这是一种智能事件转移服务,它结合了多种机器学习技术-梯度提升分类器,聚类方法和深度神经网络-用于推荐负责团队对事件进行分类。在Microsoft Azure上的真实事件上的实验结果表明,我们的服务达到了82.9%的F1分数。对于高度受影响的事件,DeepTriage的F1得分为76.3% -91.3%。我们已经应用了最佳实践和最先进的框架来扩展DeepTriage,以处理所有云服务的事件路由。DeepTriage自2017年10月以来一直部署在Azure中,每天有数千个团队使用。

英文摘要:

As cloud services are growing and generating high revenues, the cost of downtime in these services is becoming significantly expensive. To reduce loss and service downtime, a critical primary step is to execute incident triage, the process of assigning a service incident to the correct responsible team, in a timely manner. An incorrect assignment risks additional incident reroutings and increases its time to mitigate by 10x. However, automated incident triage in large cloud services faces many challenges: (1) a highly imbalanced incident distribution from a large number of teams, (2) wide variety in formats of input data or data sources, (3) scaling to meet production-grade requirements, and (4) gaining engineers' trust in using machine learning recommendations. To address these challenges, we introduce DeepTriage, an intelligent incident transfer service combining multiple machine learning techniques - gradient boosted classifiers, clustering methods, and deep neural networks - in an ensemble to recommend the responsible team to triage an incident. Experimental results on real incidents in Microsoft Azure show that our service achieves 82.9% F1 score. For highly impacted incidents, DeepTriage achieves F1 score from 76.3% - 91.3%. We have applied best practices and state-of-the-art frameworks to scale DeepTriage to handle incident routing for all cloud services. DeepTriage has been deployed in Azure since October 2017 and is used by thousands of teams daily.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑