arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.02502cs.DB
收藏

数据独立性的发现算法

An Algorithm for the Discovery of Independence from Data

Miika Hannula, Bor-Kuan Song, Sebastian Link

摘要:

多年来,独立性一直被认为是许多学科的重要概念。然而,我们提出了第一个研究,调查发现问题的独立性数据。独立性(英语:Independence)是两组列之间的一个语句,表示对于表中的每两行,表中也有一行与第一组列上的第一行和第二组列上的第二行重合。我们表明,问题的决定是否有一个独立的声明,持有一个给定的表不仅是NP-完全的,但$W[3]$ -完全的,可以说是最自然的参数,即它的arity。我们建立了第一个算法,发现所有的独立性声明,保持在一个给定的表。我们在实验中与基准数据表明,我们的算法表现良好,我们的硬度结果所建立的限制内。在实践中,确定独立性声明在给定表中的比率通常是有用的。为此,我们表明,我们对独立性的处理和算法的设计使我们能够将我们的发现扩展到近似独立性。在我们的最后的实验中,我们提供了一些见解之间的权衡运行时间和近似比。当然,比率越小,独立性陈述就越接近,发现所有这些陈述所需的时间也就越长。虽然这项研究建立了第一个洞察发现独立于数据的计算特性,我们希望开始研究更复杂的独立性概念,包括嵌入式多值依赖关系,以及它们的上下文特定和概率变体。

英文摘要:

For years, independence has been considered as an important concept in many disciplines. Nevertheless, we present the first research that investigates the discovery problem of independence in data. In its arguably simplest form, independence is a statement between two sets of columns expressing that for every two rows in a table there is also a row in the table that coincides with the first row on the first set of columns and with the second row on the second set of columns. We show that the problem of deciding whether there is an independence statement that holds on a given table is not only NP-complete but $W[3]$-complete in its arguably most natural parameter, namely its arity. We establish the first algorithm to discover all independence statement that hold on a given table. We illustrate in experiments with benchmark data that our algorithm performs well within the limits established by our hardness results. In practice, it is often useful to determine the ratio with which independence statements hold on a given table. For that purpose, we show that our treatment of independence and the design of our algorithm enables us to extend our findings to approximate independence. In our final experiments, we provide some insight into the trade-off between run time and the approximation ratio. Naturally, the smaller the ratio, the more approximate independence statements hold, and the more time it takes to discover all of them. While this research establishes first insight into the computational properties of discovering independence from data, we hope to initiate research into more sophisticated notions of independence, including embedded multivalued dependencies, as well as their context-specific and probabilistic variants.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑