arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.23769cs.SE
收藏

机器学习代码中日志异味的实证分析

Empirical Characterization of Logging Smells in Machine Learning Code

Polytechnique Montreal(蒙特利尔理工学院)

Patrick Loic Foalem, Leuson Da Silva, Foutse Khomh, Heng Li, Ettore Merlo

AI总结:

本文通过分析444个机器学习项目,识别出12种日志异味类型,揭示日志质量对系统可重复性、可维护性和可信度的影响,并公开数据集以支持未来研究。

摘要:

日志在确保机器学习(ML)系统可重复性、可观测性和可靠性方面起着核心作用。尽管日志通常被视为良好的工程实践,但设计不佳的日志可能会对实验跟踪、安全、调试和系统性能产生负面影响。本文通过分析444个ML仓库,手动标注了2,448个日志异味实例,提出了ML特定的日志异味类型分类。我们的研究发现,日志异味在ML系统中普遍存在,并且在不同项目中频率和表现形式各异。为了评估实际相关性,我们对27名ML从业者进行了调查。大多数受访者同意了识别出的异味,并报告称几种类型,包括日志敏感数据、指标覆盖、缺失超参数日志和无上下文的日志,对可重复性、可维护性和可信度有显著影响。其他异味,如重数据日志和基于打印的日志,被认为更具上下文依赖性。我们公开发布了我们的标注数据集以支持未来研究。我们的发现突显了日志质量作为ML系统工程中关键但未被充分研究的方面,并为自动检测和修复日志问题提供了机会。

英文摘要:

Logging plays a central role in ensuring reproducibility, observability, and reliability in machine learning (ML) systems. While logging is generally considered a good engineering practice, poorly designed logging can negatively affect experiment tracking, security, debugging, and system performance. In this paper, we present an empirical study of logging smells in ML projects and propose a taxonomy of ML-specific logging smell types. We conducted a large-scale analysis of 444 ML repositories and manually labeled 2,448 instances of logging smells. Based on this analysis, we identified 12 categories of logging smells spanning security, metric management, configuration, verbosity, and context-related issues. Our results show that logging smells are widespread in ML systems and vary in frequency and manifestation across projects. To assess practical relevance, we conducted a survey with 27 ML practitioners. Most respondents agreed with the identified smells and reported that several types, including Logging Sensitive Data, Metric Overwrite, Missing Hyperparameter Logging, and Log Without Context, have a strong impact on reproducibility, maintainability, and trustworthiness. Other smells, such as Heavy Data Logging and Print-based Logging, were perceived as more context-dependent. We publicly release our labeled dataset to support future research. Our findings highlight logging quality as a critical and underexplored aspect of ML system engineering and open opportunities for automated detection and repair of logging issues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑