检索、合并、预测:利用数据湖增强表格
Retrieve, Merge, Predict: Augmenting Tables with Data Lakes
AI总结:
针对数据湖的机器学习表格增强需求,研究了检索、合并、预测三步的不同方法,基于两个数据湖实验验证了候选表检索的重要性、简单合并的高效性与树模型的抗噪性,提供可复现开源环境以推动相关研究。
摘要:
从异构表格集合(即数据湖)中进行机器学习,需要通过合并与聚合表格来组装特征。数据发现可通过自动化这些步骤,将自动机器学习(autoML)拓展至数据表场景。本文针对机器学习任务中的这类自动化表格增强方法展开深入分析,研究了三大核心步骤的不同实现方式:检索可连接表格、合并信息,以及利用生成的表格进行预测。研究使用两个数据湖:一是被广泛引用的真实数据湖Open Data US,二是我们为基准测试该数据发现任务而开发的新型半合成数据集YADL(又一个数据湖)。在两个数据湖上的系统性探究表明:1)准确检索待连接候选表格至关重要;2)简单合并方法具备高效性;3)基于树的学习器对噪声环境具有鲁棒性。我们的实验环境基于公开数据,可轻松复现,旨在推动特征工程、autoML及数据湖学习领域的更多研究。
英文摘要:
Machine-learning from a disparate set of tables, a data lake, requires assembling features by merging and aggregating tables. Data discovery can extend autoML to data tables by automating these steps. We present an in-depth analysis of such automated table augmentation for machine learning tasks, analyzing different methods for the three main steps: retrieving joinable tables, merging information, and predicting with the resultant table. We use two data lakes: Open Data US, a well-referenced real data lake, and a novel semi-synthetic dataset, YADL (Yet Another Data Lake), which we developed as a tool for benchmarking this data discovery task. Systematic exploration on both lakes outlines 1) the importance of accurately retrieving candidate tables to join, 2) the efficiency of simple merging methods, and 3) the resilience of tree-based learners to noisy conditions. Our experimental environment is easily reproducible and based on open data, to foster more research on feature engineering, autoML, and learning in data lakes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。