将结构化数据统一为图以进行数据到文本预训练
Unifying Structured Data as Graph for Data-to-Text Pre-Training
AI总结：
本文将多种结构化数据统一为图格式，提出结构增强Transformer预训练方法，通过位置矩阵和注意力矩阵编码图结构信息，在六个基准数据集上验证了有效性。
摘要：
数据到文本（D2T）生成旨在将结构化数据转化为自然语言文本。数据到文本预训练已被证明在增强D2T生成方面非常有效，并取得了令人瞩目的性能。然而，以往的预训练方法要么将结构化数据过度简化为序列而不考虑输入结构，要么针对特定数据结构（例如表格或知识图谱）设计训练目标。在本文中，我们将不同类型的结构化数据（即表格、键值数据、知识图谱）统一为图格式，并将不同的数据到文本生成任务转化为图到文本生成。为了有效利用输入图的结构信息，我们通过设计结构增强Transformer，提出了一种用于D2T生成的结构增强预训练方法。具体而言，我们为Transformer设计了一个位置矩阵，用于编码输入图中相连节点的相对位置信息。此外，我们提出了一种新的注意力矩阵，通过考虑可用的显式连接结构，将图结构纳入原始Transformer中。在六个基准数据集上的广泛实验证明了我们模型的有效性。我们的源代码可在https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/unid2t获取。
英文摘要：
Data-to-text (D2T) generation aims to transform structured data into natural language text. Data-to-text pre-training has proved to be powerful in enhancing D2T generation and yields impressive performances. However, previous pre-training methods either oversimplified structured data into a sequence without considering input structures or designed training objectives tailored for a specific data structure (e.g., table or knowledge graph). In this paper, we unify different types of structured data (i.e., table, key-value data, knowledge graph) into the graph format and cast different data-to-text generation tasks as graph-to-text generation. To effectively exploit the structural information of the input graph, we propose a structure-enhanced pre-training method for D2T generation by designing a structure-enhanced Transformer. Concretely, we devise a position matrix for the Transformer, encoding relative positional information of connected nodes in the input graph. In addition, we propose a new attention matrix to incorporate graph structures into the original Transformer by taking the available explicit connectivity structure into account. Extensive experiments on six benchmark datasets show the effectiveness of our model. Our source codes are available at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/unid2t.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
