arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.13636cs.DBcs.AI
收藏

AutoCure:面向机器学习流水线的自动化表格数据治理技术

AutoCure: Automated Tabular Data Curation Technique for ML Pipelines

Software AG(软件AG公司);Otto von Guericke University Magdeburg(马格德堡奥托·冯·格里克大学)

Mohamed Abdelaal, Rashmi Koparde, Harald Schoening

AI总结:

本文提出无需配置的 AutoCure 表格数据治理流水线,结合自适应集成错误检测与数据增强,并可集成 Auto-sklearn 等工具;对比28种传统工具组合验证其能自动提升ML模型准确率。

摘要:

机器学习算法已在自动驾驶、医疗保健和金融等多个领域日益普及。在这些领域中,数据准备仍是开发准确模型的一项重大挑战,需要投入大量专业知识和时间,才能在庞大的搜索空间中寻找合适的数据治理与转换工具。为应对这一挑战,我们提出了 AutoCure,这是一种新颖且无需配置的数据治理流水线,可提升表格数据的质量。与传统数据治理方法不同,AutoCure 通过基于集成的自适应错误检测方法和数据增强模块,综合提高干净数据部分的密度。在实践中,AutoCure 可与 Auto-sklearn、H2O 和 TPOT 等开源工具集成,以推动机器学习的民主化。作为概念验证,我们将 AutoCure 与 28 种传统数据治理工具组合进行了对比评估,结果表明其在无需用户干预的情况下具有更优的性能和预测准确率。我们的评估表明,AutoCure 是一种自动化数据准备并提高机器学习模型准确率的有效方法。

英文摘要:

Machine learning algorithms have become increasingly prevalent in multiple domains, such as autonomous driving, healthcare, and finance. In such domains, data preparation remains a significant challenge in developing accurate models, requiring significant expertise and time investment to search the huge search space of well-suited data curation and transformation tools. To address this challenge, we present AutoCure, a novel and configuration-free data curation pipeline that improves the quality of tabular data. Unlike traditional data curation methods, AutoCure synthetically enhances the density of the clean data fraction through an adaptive ensemble-based error detection method and a data augmentation module. In practice, AutoCure can be integrated with open source tools, e.g., Auto-sklearn, H2O, and TPOT, to promote the democratization of machine learning. As a proof of concept, we provide a comparative evaluation of AutoCure against 28 combinations of traditional data curation tools, demonstrating superior performance and predictive accuracy without user intervention. Our evaluation shows that AutoCure is an effective approach to automating data preparation and improving the accuracy of machine learning models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑