Explain3D:解释不相交数据集中的分歧
Explain3D: Explaining Disagreements in Disjoint Datasets
摘要:
数据在应用程序、分析过程和人类活动的许多方面发挥着重要作用。随着数据规模和复杂性的增长,我们迫切需要能够促进对数据相关操作的理解和解释的工具。关于解释的数据管理研究集中在这样一个假设上,即数据驻留在一个公共模式下的单个数据集中。但今天的数据的现实是,它经常是未集成的,来自不同的来源,具有不同的模式。当不同的数据集对语义相似的问题提供不同的答案时,理解差异的原因是具有挑战性的,并且无法通过现有的单数据集解决方案来处理。在本文中,我们提出了Explain 3D,一个用于解释不相交数据集(3D)之间的分歧的框架。Explain 3D的重点是确定两个语义相似的查询在两个数据集上操作的结果存在差异的原因,这些数据集具有潜在的不同模式。我们的框架利用查询执行语义映射,在其出处的相关部分,在这个映射点的差异查询的差异的原因。利用查询使Explain 3D比传统的模式匹配和记录链接技术更有优势,这些技术与查询无关。我们的工作有以下几个方面的贡献:(1)我们形式化的问题,推导出最佳的解释的语义相似的查询结果的差异不相交的数据集。(2)我们设计了一个解决最优解释问题的三阶段框架。(3)我们开发了一个智能分区优化器,提高了几个数量级的框架的效率。(4)~我们使用真实世界和合成数据进行实验,以证明Explain 3D可以有效地获得精确的解释。
英文摘要:
Data plays an important role in applications, analytic processes, and many aspects of human activity. As data grows in size and complexity, we are met with an imperative need for tools that promote understanding and explanations over data-related operations. Data management research on explanations has focused on the assumption that data resides in a single dataset, under one common schema. But the reality of today's data is that it is frequently un-integrated, coming from different sources with different schemas. When different datasets provide different answers to semantically similar questions, understanding the reasons for the discrepancies is challenging and cannot be handled by the existing single-dataset solutions. In this paper, we propose Explain3D, a framework for explaining the disagreements across disjoint datasets (3D). Explain3D focuses on identifying the reasons for the differences in the results of two semantically similar queries operating on two datasets with potentially different schemas. Our framework leverages the queries to perform a semantic mapping across the relevant parts of their provenance; discrepancies in this mapping point to causes of the queries' differences. Exploiting the queries gives Explain3D an edge over traditional schema matching and record linkage techniques, which are query-agnostic. Our work makes the following contributions: (1) We formalize the problem of deriving optimal explanations for the differences of the results of semantically similar queries over disjoint datasets. (2) We design a 3-stage framework for solving the optimal explanation problem. (3) We develop a smart-partitioning optimizer that improves the efficiency of the framework by orders of magnitude. (4)~We experiment with real-world and synthetic data to demonstrate that Explain3D can derive precise explanations efficiently.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。