arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.08594cs.AIcs.DB
收藏

表格数据的语义标注

Semantic Annotation for Tabular Data

IBM Research AI(IBM研究院人工智能);University of Massachusetts, Amherst(马萨诸塞大学阿默斯特分校)

Udayan Khurana, Sainyam Galhotra

摘要:

在机器学习中,从数据集成、清洗、搜索到特征工程和模型构建的许多应用中,对表格数据中的列的语义概念的检测特别感兴趣。最近,一些作品提出了基于监督学习或启发式模式的语义类型标注方法。两者都有缺点,使他们无法概括大量的概念或例子。许多基于神经网络的方法也存在可扩展性问题。此外,没有一个已知的方法对数值数据很好地工作。我们提出了$C^2$,一列概念映射,是基于最大似然估计方法通过合奏。除了两个流行的知识图谱之外,它还能够有效地利用大量的(尽管有些嘈杂)公开可用的表格语料库来对结构化数据执行有效且高效的概念预测。我们证明了$C^2$的有效性,在9个数据集上的可用技术,迄今为止最全面的比较。

英文摘要:

Detecting semantic concept of columns in tabular data is of particular interest to many applications ranging from data integration, cleaning, search to feature engineering and model building in machine learning. Recently, several works have proposed supervised learning-based or heuristic pattern-based approaches to semantic type annotation. Both have shortcomings that prevent them from generalizing over a large number of concepts or examples. Many neural network based methods also present scalability issues. Additionally, none of the known methods works well for numerical data. We propose $C^2$, a column to concept mapper that is based on a maximum likelihood estimation approach through ensembles. It is able to effectively utilize vast amounts of, albeit somewhat noisy, openly available table corpora in addition to two popular knowledge graphs to perform effective and efficient concept prediction for structured data. We demonstrate the effectiveness of $C^2$ over available techniques on 9 datasets, the most comprehensive comparison on this topic so far.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑