在线服务中的非结构化查询综述
Summarizing Unstructured Logs in Online Services
摘要:
数据库是管理大规模在线服务最有价值的数据源之一。在检测/诊断/预测故障后,操作员仍然必须检查原始日志,以便在采取行动之前获得汇总视图。但是,手动或基于规则的日志摘要已经变得低效和无效。在这项工作中,我们提出了LogSummary,一个自动的,无监督的端到端的在线服务日志摘要框架。LogSummary获取给定日志序列的重要日志的汇总三元组。它集成了一种新的信息抽取方法,同时考虑到语义信息和领域知识,一个新的三重排序方法,使用从所有日志的全局知识。由于缺乏公开的日志摘要黄金标准,我们手动标记了四个开源日志数据集的摘要,并将其公开。对这些数据集的评估以及对真实日志的案例研究表明,LogSummary生成了高度代表性的摘要(平均ROUGE F1得分为0.741)。我们已经将LogSummary打包到一个开源工具包中,希望它能够为未来的NLP支持的摘要工作带来好处。
英文摘要:
Logs are one of the most valuable data sources for managing large-scale online services. After a failure is detected/diagnosed/predicted, operators still have to inspect the raw logs to gain a summarized view before take actions. However, manual or rule-based log summarization has become inefficient and ineffective. In this work, we propose LogSummary, an automatic, unsupervised end-to-end log summarization framework for online services. LogSummary obtains the summarized triples of important logs for a given log sequence. It integrates a novel information extraction method taking both semantic information and domain knowledge into consideration, with a new triple ranking approach using the global knowledge learned from all logs. Given the lack of a publicly-available gold standard for log summarization, we have manually labelled the summaries of four open-source log datasets and made them publicly available. The evaluation on these datasets as well as the case studies on real-world logs demonstrate that LogSummary produces a highly representative (average ROUGE F1 score of 0.741) summaries. We have packaged LogSummary into an open-source toolkit and hope that it can benefit for future NLP-powered summarization works.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。