arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.05308cs.DB
收藏

以目标精度进行数据清理:值规范化的案例研究

Toward Data Cleaning with a Target Accuracy: A Case Study for Value Normalization

Adel Ardalan, Derek Paulsen, Amanpreet Singh Saini, Walter Cai, AnHai Doan

摘要:

许多应用程序需要以目标精度清理数据。据我们所知,这个问题还没有得到深入研究。在本文中,我们迈出了解决这个问题的第一步,我们专注于值规范化(VN),用一个唯一的字符串替换所有引用同一实体的字符串的问题。VN是无处不在的,我们经常希望以100%的准确度进行VN。这通常是通过自动聚类字符串,然后要求用户验证和清理集群,直到达到100%的准确性来完成的。这种解决方案有很大的局限性。它没有告诉用户如何验证和清理集群。这一部分通常也需要很多时间,例如,天此外,对于多个用户来说,没有有效的方式来协作地验证和清理。在本文中,我们解决这些挑战。总的来说,我们的工作通过引入一个新的清洗问题和描述一个有前途的解决方案模板,推进了数据清洗的最新技术水平。

英文摘要:

Many applications need to clean data with a target accuracy. As far as we know, this problem has not been studied in depth. In this paper we take the first step toward solving it. We focus on value normalization (VN), the problem of replacing all string that refer to the same entity with a unique string. VN is ubiquitous, and we often want to do VN with 100% accuracy. This is typically done today in industry by automatically clustering the strings then asking a user to verify and clean the clusters, until reaching 100% accuracy. This solution has significant limitations. It does not tell the users how to verify and clean the clusters. This part also often takes a lot of time, e.g., days. Further, there is no effective way for multiple users to collaboratively verify and clean. In this paper we address these challenges. Overall, our work advances the state of the art in data cleaning by introducing a novel cleaning problem and describing a promising solution template.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑