缺失数据自动填充
Missing Data Infill with Automunge
摘要：
缺失数据是数据科学实践中的一个根本障碍。本文调查了Automunge开源python库平台中用于表格数据预处理的一些插补惯例，包括“ML填充”，其中自动ML模型从训练集的分区提取中训练目标特征。进行了一系列验证实验，以基准插补场景对下游模型性能，其中发现对于给定的基准集，在许多情况下，ML填充优于数字和分类目标特征，并且在其他插补场景的噪声分布内处于最小值。证据还表明，通过添加具有布尔整数标记的支撑柱来补充ML填充物，该布尔整数标记指示填充物的存在通常有利于下游模型性能。我们认为这些结果足以推荐默认为ML填充用于表格学习，并进一步推荐用支持列补充插补，支持列表示填充的存在，每个都可以在Automunge库中通过按钮操作准备。我们的贡献包括一个用于Python生态系统中表格学习的自动ML派生的缺失数据填补库，完全集成到一个具有广泛特征转换库的预处理平台中，具有一个新的生产友好实现，该实现将填补模型基于指定的训练集，以实现对额外数据的一致性。
英文摘要：
Missing data is a fundamental obstacle in the practice of data science. This paper surveys a few conventions for imputation as available in the Automunge open source python library platform for tabular data preprocessing, including "ML infill" in which auto ML models are trained for target features from partitioned extracts of a training set. A series of validation experiments were performed to benchmark imputation scenarios towards downstream model performance, in which it was found for the given benchmark sets that in many cases ML infill outperformed for both numeric and categoric target features, and was otherwise at minimum within noise distributions of the other imputation scenarios. Evidence also suggested supplementing ML infill with the addition of support columns with boolean integer markers signaling presence of infill was usually beneficial to downstream model performance. We consider these results sufficient to recommend defaulting to ML infill for tabular learning, and further recommend supplementing imputations with support columns signaling presence of infill, each as can be prepared with push-button operation in the Automunge library. Our contributions include an auto ML derived missing data imputation library for tabular learning in the python ecosystem, fully integrated into a preprocessing platform with an extensive library of feature transformations, with a novel production friendly implementation that bases imputation models on a designated train set for consistent basis towards additional data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
