arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.07323cs.IRcs.LG
收藏

FORTAP:使用公式进行数值推理感知表预训练

FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining

Shanghai Jiao Tong University(上海交通大学);Microsoft Research Asia(微软亚洲研究院);Fudan University(复旦大学)

Zhoujun Cheng, Haoyu Dong, Ran Jia, Pengfei Wu, Shi Han, Fan Cheng, Dongmei Zhang

摘要:

表格存储了丰富的数值数据,但对表格进行数值推理仍然是一个挑战。在本文中,我们发现,电子表格公式,它执行计算的数值在表格中,自然是一个强大的监督数值推理。更重要的是,大量的电子表格与专家制定的公式可以在网上,可以很容易地获得。FORTAP是第一种利用大型电子表格公式语料库进行数字推理感知表预训练的方法。我们设计了两个公式预训练任务来明确地指导FORTAP学习半结构化表中的数值引用和计算。FORTAP在两个具有代表性的下游任务(细胞类型分类和公式预测)上取得了最先进的结果,显示了数字推理感知预训练的巨大潜力。

英文摘要:

Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, which performs calculations on numerical values in tables, is naturally a strong supervision of numerical reasoning. More importantly, large amounts of spreadsheets with expert-made formulae are available on the web and can be obtained easily. FORTAP is the first method for numerical-reasoning-aware table pretraining by leveraging large corpus of spreadsheet formulae. We design two formula pretraining tasks to explicitly guide FORTAP to learn numerical reference and calculation in semi-structured tables. FORTAP achieves state-of-the-art results on two representative downstream tasks, cell type classification and formula prediction, showing great potential of numerical-reasoning-aware pretraining.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑