arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.09162cs.CLcs.LG
收藏

使用TabT5进行表格到文本生成和预训练

Table-To-Text generation and pre-training with TabT5

Google Research(谷歌研究院)

Ewa Andrejczuk, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Yasemin Altun

摘要:

仅编码器的Transformer模型已经成功地应用于不同的表理解任务,如在TAPAS(Herzig等人,2020年)。这些架构的一个主要限制是,它们被约束到分类类任务,如细胞选择或蕴涵检测。我们提出了TABT 5,一个编码器-解码器模型,它根据表格和文本输入生成自然语言文本。TABT 5通过合并解码器组件克服了仅编码器的限制,并利用具有表特定嵌入和预训练的输入结构。TABT 5在多个领域取得了最新的成果,包括电子表格公式预测的序列准确性提高了15%,QA的序列准确性提高了2.5%,数据到文本的生成在BLEU中提高了2.5%。

英文摘要:

Encoder-only transformer models have been successfully applied to different table understanding tasks, as in TAPAS (Herzig et al., 2020). A major limitation of these architectures is that they are constrained to classification-like tasks such as cell selection or entailment detection. We present TABT5, an encoder-decoder model that generates natural language text based on tables and textual inputs. TABT5 overcomes the encoder-only limitation by incorporating a decoder component and leverages the input structure with table specific embeddings and pre-training. TABT5 achieves new state-of-the-art results on several domains, including spreadsheet formula prediction with a 15% increase in sequence accuracy, QA with a 2.5% increase in sequence accuracy and data-to-text generation with a 2.5% increase in BLEU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑