对AIOps模型的一致解释
Towards a consistent interpretation of AIOps models
摘要:
IT运营人工智能(AIOps)已在组织的各种任务中采用,包括解释模型以识别服务故障指标。为了避免误导从业人员,AIOps模型的解释应该是一致的(即,相同任务上的不同AIOps模型在特征重要性上彼此一致)。然而,许多AIOps研究在推导解释时违反了机器学习社区中的既定实践,例如解释性能次优的模型,尽管尚未研究这种违反对解释一致性的影响。在本文中,我们沿着三个维度:内部一致性,外部一致性和时间一致性研究AIOps模型解释的一致性。我们对两个AIOps任务进行了案例研究:预测Google集群作业失败和Backblaze硬盘驱动器故障。我们发现,学习者的随机性,超参数调整,和数据采样应加以控制,以产生一致的解释。与低性能模型相比,AUC大于0.75的AIOps模型产生更一致的解释。最后,使用滑动窗口或完整历史方法构建的AIOps模型与整个数据集中呈现的趋势具有最一致的解释。我们的研究为从业者获得一致的AIOps模型解释提供了有价值的指导方针。
英文摘要:
Artificial Intelligence for IT Operations (AIOps) has been adopted in organizations in various tasks, including interpreting models to identify indicators of service failures. To avoid misleading practitioners, AIOps model interpretations should be consistent (i.e., different AIOps models on the same task agree with one another on feature importance). However, many AIOps studies violate established practices in the machine learning community when deriving interpretations, such as interpreting models with suboptimal performance, though the impact of such violations on the interpretation consistency has not been studied. In this paper, we investigate the consistency of AIOps model interpretation along three dimensions: internal consistency, external consistency, and time consistency. We conduct a case study on two AIOps tasks: predicting Google cluster job failures, and Backblaze hard drive failures. We find that the randomness from learners, hyperparameter tuning, and data sampling should be controlled to generate consistent interpretations. AIOps models with AUCs greater than 0.75 yield more consistent interpretation compared to low-performing models. Finally, AIOps models that are constructed with the Sliding Window or Full History approaches have the most consistent interpretation with the trends presented in the entire datasets. Our study provides valuable guidelines for practitioners to derive consistent AIOps model interpretation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。