arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.01765cs.LGcs.DB
收藏

数据清洗与机器学习:一项系统性文献综述

Data Cleaning and Machine Learning: A Systematic Literature Review

Polytechnique Montréal(蒙特利尔理工学院)

Pierre-Olivier Côté, Amin Nikanjam, Nafisa Ahmed, Dmytro Humeniuk, Foutse Khomh

AI总结:

本文系统综述了2016至2022年间101篇文献,总结了机器学习与数据清洗的双重关系,涵盖多种数据清洗活动并提供24条未来研究建议。

摘要:

背景:机器学习(ML)被集成到越来越多的系统中以用于各种应用。由于ML模型的性能高度依赖于其训练数据的质量,因此人们对检测和修复数据错误(即数据清洗)的方法越来越感兴趣。研究人员也在探索如何将ML用于数据清洗;从而在ML和数据清洗之间创建了一种双重关系。据我们所知,目前没有研究全面综述这种关系。目标:本文的目标有两个方面。首先,它旨在总结用于ML的数据清洗和用于数据清洗的ML的最新方法。其次,它提供了对未来工作的建议。方法:我们对2016年至2022年(含)期间发表的论文进行了系统性文献综述。我们识别了针对ML和用于ML的不同类型的数据清洗活动:特征清洗、标签清洗、实体匹配、异常值检测、插补和整体数据清洗。结果:我们总结了101篇涵盖各种数据清洗活动的论文的内容,并提供了24条未来工作建议。我们的综述强调了许多有前景的、可进一步扩展的数据清洗技术。结论:我们相信我们对文献的综述将帮助社区开发出更好的数据清洗方法。

英文摘要:

Context: Machine Learning (ML) is integrated into a growing number of systems for various applications. Because the performance of an ML model is highly dependent on the quality of the data it has been trained on, there is a growing interest in approaches to detect and repair data errors (i.e., data cleaning). Researchers are also exploring how ML can be used for data cleaning; hence creating a dual relationship between ML and data cleaning. To the best of our knowledge, there is no study that comprehensively reviews this relationship. Objective: This paper's objectives are twofold. First, it aims to summarize the latest approaches for data cleaning for ML and ML for data cleaning. Second, it provides future work recommendations. Method: We conduct a systematic literature review of the papers published between 2016 and 2022 inclusively. We identify different types of data cleaning activities with and for ML: feature cleaning, label cleaning, entity matching, outlier detection, imputation, and holistic data cleaning. Results: We summarize the content of 101 papers covering various data cleaning activities and provide 24 future work recommendations. Our review highlights many promising data cleaning techniques that can be further extended. Conclusion: We believe that our review of the literature will help the community develop better approaches to clean data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑