arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.07489cs.CL
收藏

图-序列神经机器翻译

Graph-to-Sequence Neural Machine Translation

Shanghai Jiao Tong University(上海交通大学);National Institute of Information and Communications Technology (NICT)(日本信息通信研究机构)

Sufeng Duan, Hai Zhao, Rui Wang

摘要:

神经机器翻译(Neural Machine Translation,NMT)通常采用seq 2seq的学习方式,将源语句或目标语句看作是一个线性的词序列,可以看作是图的一种特殊情况,以序列中的词为节点,词与词之间的关系为边。针对现有的NMT模型或多或少地以潜在的方式捕获序列中的图信息的问题,提出了一种图到序列模型,该模型便于显式地捕获图信息。具体地说,我们提出了一个基于图的SAN的NMT模型称为Graph-Transformer,通过捕获每一层中不同阶数的子图的信息。子图按顺序分组,每组子图分别反映词与词之间不同程度的依存关系。对于融合子图表示,我们经验性地探索了三种方法,这些方法对不同阶数的不同子图组进行加权。在WMT 14英德和IWSLT 14德英数据集上的实验结果表明,该方法能有效地提高Transformer的性能,在WMT 14英德数据集上提高了1.1个BLEU点,在IWSLT 14德英数据集上提高了1.0个BLEU点。

英文摘要:

Neural machine translation (NMT) usually works in a seq2seq learning way by viewing either source or target sentence as a linear sequence of words, which can be regarded as a special case of graph, taking words in the sequence as nodes and relationships between words as edges. In the light of the current NMT models more or less capture graph information among the sequence in a latent way, we present a graph-to-sequence model facilitating explicit graph information capturing. In detail, we propose a graph-based SAN-based NMT model called Graph-Transformer by capturing information of subgraphs of different orders in every layers. Subgraphs are put into different groups according to their orders, and every group of subgraphs respectively reflect different levels of dependency between words. For fusing subgraph representations, we empirically explore three methods which weight different groups of subgraphs of different orders. Results of experiments on WMT14 English-German and IWSLT14 German-English show that our method can effectively boost the Transformer with an improvement of 1.1 BLEU points on WMT14 English-German dataset and 1.0 BLEU points on IWSLT14 German-English dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑