Groot：一种基于事件图的工业环境根本原因分析方法
Groot: An Event-graph-based Approach for Root Cause Analysis in Industrial Settings
摘要：
对于大规模分布式系统，有效地诊断事件的根本原因以保持高系统可用性至关重要。微服务架构的最新发展带来了三大挑战（即，操作、系统规模和监控复杂性）到工业环境中的根本原因分析（RCA）。为了应对这些挑战，在本文中，我们提出了Groot，一个基于事件图的RCA方法。Groot基于事件构建实时因果关系图，总结所分析系统中各种类型的指标、日志和活动。此外，为了整合站点可靠性工程（SRE）工程师的领域知识，Groot可以使用用户定义的事件和特定于领域的规则进行定制。目前，Groot支持5,000个真实生产服务中的RCA，并被SRE团队积极使用，该团队是一个全球电子商务系统，每年为超过1.85亿活跃买家提供服务。在15个月的时间里，我们收集了一个数据集，其中包含952个真实生产事件的标记根本原因，以供评估。评估结果表明，Groot能够达到95%的top-3准确率和78%的top-1准确率。为了分享我们在工业环境中部署和采用RCA的经验，我们进行了调查，以显示Grootfindit的用户非常有用且易于使用。我们还分享了在生产环境中部署和采用Groot解决RCA问题的经验教训。
英文摘要：
For large-scale distributed systems, it's crucial to efficiently diagnose the root causes of incidents to maintain high system availability. The recent development of microservice architecture brings three major challenges (i.e., operation, system scale, and monitoring complexities) to root cause analysis (RCA) in industrial settings. To tackle these challenges, in this paper, we present Groot, an event-graph-based approach for RCA. Groot constructs a real-time causality graph based on events that summarize various types of metrics, logs, and activities in the system under analysis. Moreover, to incorporate domain knowledge from site reliability engineering (SRE) engineers, Groot can be customized with user-defined events and domain-specific rules. Currently, Groot supports RCA among 5,000 real production services and is actively used by the SRE teamin a global e-commerce system serving more than 185 million active buyers per year. Over 15 months, we collect a data setcontaining labeled root causes of 952 real production incidents for evaluation. The evaluation results show that Groot is able to achieve 95% top-3 accuracy and 78% top-1 accuracy. To share our experience in deploying and adopting RCA in industrial settings, we conduct survey to show that users of Grootfindit helpful and easy to use. We also share the lessons learnedfrom deploying and adopting Grootto solve RCA problems inproduction environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
