RTClean:基于实时的OFD的上下文感知表格数据清理
RTClean: Context-aware Tabular Data Cleaning using Real-time OFDs
摘要:
如今,机器学习在开发大量应用中发挥着关键作用,例如,灵巧家庭、智能医疗辅助、自动驾驶。这些应用的一个主要挑战是保持高质量的训练和服务数据。然而,现有的数据清洗方法不能利用上下文信息。因此,它们通常无法跟踪数据分布或相关误差分布的变化。为了克服这些局限性,我们在本文中,介绍了一种新的方法,自动化的表格数据清洗的动态功能依赖规则提取的活上下文模型供电。作为概念验证,我们创建了一个灵巧家庭用例来收集数据,同时保留上下文信息。使用两个不同的数据集,我们的评估表明,建议的清洗方法优于一组基线方法的检测和修复精度。
英文摘要:
Nowadays, machine learning plays a key role in developing plenty of applications, e.g., smart homes, smart medical assistance, and autonomous driving. A major challenge of these applications is preserving high quality of the training and the serving data. Nevertheless, existing data cleaning methods cannot exploit context information. Thus, they usually fail to track shifts in the data distributions or the associated error profiles. To overcome these limitations, we introduce, in this paper, a novel method for automated tabular data cleaning powered by dynamic functional dependency rules extracted from a live context model. As a proof of concept, we create a smart home use case to collect data while preserving the context information. Using two different data sets, our evaluations show that the proposed cleaning method outperforms a set of baseline methods in terms of the detection and repair accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。