arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.07736cs.DBcs.LG
收藏

Observatory:表征关系表的嵌入

Observatory: Characterizing Embeddings of Relational Tables

University of Michigan(密歇根大学);University of Amsterdam(阿姆斯特丹大学)

Tianji Cong, Madelon Hulsebos, Zhenjie Sun, Paul Groth, H. V. Jagadish

AI总结:

针对表嵌入模型选型依赖试错的问题,提出可系统分析关系表嵌入的Observatory框架,从8项属性量化评估9个模型,揭示其优缺点以指导选型与新模型开发。

摘要:

语言模型和专用表嵌入模型近期在诸多表格数据任务上展现出强劲性能。研究人员与从业者迫切希望在众多新应用场景中运用这些模型;但由于对这些模型及其生成的表表示的优缺点了解有限,为给定任务寻找合适模型的过程只能依赖试错。目前迫切需要全面了解这些模型,以减少下游应用中的低效问题与失败情况。\n 为满足这一需求,我们提出Observatory——一个用于系统分析关系表嵌入表示的正式框架。受关系数据模型的不变量以及数据分布相关统计考量的启发,我们定义了8项基本属性,以及相应的量化指标,以表征表嵌入在这些属性上的表现。基于这些属性,我们构建了一个可扩展框架,用于评估语言模型与表嵌入模型。我们收集并整合了一套数据集,利用Observatory分析了9个此类模型。我们的分析揭示了表的学习表示的优缺点。例如,我们发现部分模型对列顺序等表结构敏感,函数依赖极少在嵌入中得到体现,专用表嵌入模型的样本保真度相对较低。这些发现能帮助研究人员与从业者更好地预判模型行为,为下游任务选择合适的模型,同时也能为研究人员开发新模型提供指导。

英文摘要:

Language models and specialized table embedding models have recently demonstrated strong performance on many tasks over tabular data. Researchers and practitioners are keen to leverage these models in many new application contexts; but limited understanding of the strengths and weaknesses of these models, and the table representations they generate, makes the process of finding a suitable model for a given task reliant on trial and error. There is an urgent need to gain a comprehensive understanding of these models to minimize inefficiency and failures in downstream usage. To address this need, we propose Observatory, a formal framework to systematically analyze embedding representations of relational tables. Motivated both by invariants of the relational data model and by statistical considerations regarding data distributions, we define eight primitive properties, and corresponding measures to quantitatively characterize table embeddings for these properties. Based on these properties, we define an extensible framework to evaluate language and table embedding models. We collect and synthesize a suite of datasets and use Observatory to analyze nine such models. Our analysis provides insights into the strengths and weaknesses of learned representations over tables. We find, for example, that some models are sensitive to table structure such as column order, that functional dependencies are rarely reflected in embeddings, and that specialized table embedding models have relatively lower sample fidelity. Such insights help researchers and practitioners better anticipate model behaviors and select appropriate models for their downstream tasks, while guiding researchers in the development of new models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑