arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.09263cs.CLcs.AIcs.DB
收藏

Table-GPT:面向多样表格任务的表格微调GPT

Table-GPT: Table-tuned GPT for Diverse Table Tasks

Microsoft Corporation(微软公司)

Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, Surajit Chaudhuri

AI总结:

本文提出“表格微调”范式,通过真实表格合成的任务微调GPT-3.5等语言模型,得到Table-GPT,在多样表格任务上优于原模型,并具备强泛化能力。

摘要:

语言模型,如GPT-3.5和ChatGPT,展现出遵循多样化人类指令并执行广泛任务的卓越能力。然而,当我们通过一系列基础表格理解任务来探测语言模型时,观察到当今的语言模型在许多与表格相关的任务中仍表现欠佳,这可能是因为它们主要在“一维”自然语言文本上进行预训练,而关系表格是“二维”对象。在本工作中,我们提出了一种新的“表格微调”范式,即使用从真实表格合成的多样化表格任务作为训练数据,继续训练/微调像GPT-3.5和ChatGPT这样的语言模型,旨在增强语言模型理解表格和执行表格任务的能力。我们表明,由此得到的Table-GPT模型展现出:(1)更好的表格理解能力,在广泛的表格任务(包括未见的保留任务)上持续优于原始的GPT-3.5和ChatGPT;(2)强大的泛化能力,能够以类似于GPT-3.5和ChatGPT的方式响应多样化的人类指令以执行新的表格任务。

英文摘要:

Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on \emph{one-dimensional} natural-language texts, whereas relational tables are \emph{two-dimensional} objects. In this work, we propose a new "\emph{table-tuning}" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better \emph{table-understanding} capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong \emph{generalizability}, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑