企业系统中日志放置推荐的探索性研究
An Exploratory Study of Log Placement Recommendation in an Enterprise System
摘要:
日志记录是一种开发实践,在复杂系统的操作和监控中发挥着重要作用。开发人员在源代码中放置日志语句,并使用日志数据来了解系统在生产中的行为。不幸的是,在开发过程中预测在哪里进行日志记录是一个挑战。以前的研究表明,尽管数据不平衡,但利用机器学习来推荐日志放置是可行的,因为日志记录只是整个代码库的一小部分。然而,这些技术如何应用于行业环境仍然是未知的,而且对不平衡数据和抽样技术的影响知之甚少。本文研究了大型支付公司Adyen代码库中的日志放置问题。我们分析了34,526个Java文件和309,527个方法,这些方法总计为+2M Sort。我们基于代码度量系统地测量了五个模型的有效性,探索了采样技术的效果,了解了模型认为哪些特征与预测相关,并评估了我们是否可以利用来自29个Apache项目的388,086种方法来了解在行业环境中登录的位置。我们最好的模型达到了79%的平衡准确率,81%的精确率,60%的召回率。虽然抽样技术提高了召回率,但它们以高昂的成本惩罚了精确度。使用开放源数据的实验在Adyen的测试集上产生了表现不佳的模型;然而,由于它们的误报率低,它们是有用的。我们的支持脚本和工具可供社区使用。
英文摘要:
Logging is a development practice that plays an important role in the operations and monitoring of complex systems. Developers place log statements in the source code and use log data to understand how the system behaves in production. Unfortunately, anticipating where to log during development is challenging. Previous studies show the feasibility of leveraging machine learning to recommend log placement despite the data imbalance since logging is a fraction of the overall code base. However, it remains unknown how those techniques apply to an industry setting, and little is known about the effect of imbalanced data and sampling techniques. In this paper, we study the log placement problem in the code base of Adyen, a large-scale payment company. We analyze 34,526 Java files and 309,527 methods that sum up +2M SLOC. We systematically measure the effectiveness of five models based on code metrics, explore the effect of sampling techniques, understand which features models consider to be relevant for the prediction, and evaluate whether we can exploit 388,086 methods from 29 Apache projects to learn where to log in an industry setting. Our best performing model achieves 79% of balanced accuracy, 81% of precision, 60% of recall. While sampling techniques improve recall, they penalize precision at a prohibitive cost. Experiments with open-source data yield under-performing models over Adyen's test set; nevertheless, they are useful due to their low rate of false positives. Our supporting scripts and tools are available to the community.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。