arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2012.08938cs.SEcs.NI
收藏

在线服务中的非结构化查询综述

Summarizing Unstructured Logs in Online Services

Weibin Meng, Federico Zaiter, Yuheng Huang, Ying Liu, Shenglin Zhang, Yuzhe Zhang, Yichen Zhu, Tianke Zhang, En Wang, Zuomin Ren, Feng Wang, Shimin Tao, Dan Pei

摘要:

数据库是管理大规模在线服务最有价值的数据源之一。在检测/诊断/预测故障后,操作员仍然必须检查原始日志,以便在采取行动之前获得汇总视图。但是,手动或基于规则的日志摘要已经变得低效和无效。在这项工作中,我们提出了LogSummary,一个自动的,无监督的端到端的在线服务日志摘要框架。LogSummary获取给定日志序列的重要日志的汇总三元组。它集成了一种新的信息抽取方法,同时考虑到语义信息和领域知识,一个新的三重排序方法,使用从所有日志的全局知识。由于缺乏公开的日志摘要黄金标准,我们手动标记了四个开源日志数据集的摘要,并将其公开。对这些数据集的评估以及对真实日志的案例研究表明,LogSummary生成了高度代表性的摘要(平均ROUGE F1得分为0.741)。我们已经将LogSummary打包到一个开源工具包中,希望它能够为未来的NLP支持的摘要工作带来好处。

英文摘要:

Logs are one of the most valuable data sources for managing large-scale online services. After a failure is detected/diagnosed/predicted, operators still have to inspect the raw logs to gain a summarized view before take actions. However, manual or rule-based log summarization has become inefficient and ineffective. In this work, we propose LogSummary, an automatic, unsupervised end-to-end log summarization framework for online services. LogSummary obtains the summarized triples of important logs for a given log sequence. It integrates a novel information extraction method taking both semantic information and domain knowledge into consideration, with a new triple ranking approach using the global knowledge learned from all logs. Given the lack of a publicly-available gold standard for log summarization, we have manually labelled the summaries of four open-source log datasets and made them publicly available. The evaluation on these datasets as well as the case studies on real-world logs demonstrate that LogSummary produces a highly representative (average ROUGE F1 score of 0.741) summaries. We have packaged LogSummary into an open-source toolkit and hope that it can benefit for future NLP-powered summarization works.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑