arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1811.04716cs.CL
收藏

多源Transformer解码器的输入组合策略

Input Combination Strategies for Multi-Source Transformer Decoder

Charles University(查理大学);Institute of Formal and Applied Linguistics(形式与应用语言学研究所)

Jindřich Libovický, Jindřich Helcl, David Mareček

摘要:

在多源序列到序列任务中,注意力机制可以以多种方式建模。这个主题已经在循环体系结构上进行了深入的研究。在本文中,我们将以前的工作扩展到Transformer架构中的编码器-解码器注意力。我们提出了四种不同的输入组合策略的编码器-解码器的注意力:串行,并行,平面和层次。我们评估我们的方法多模态翻译和翻译任务与多个源语言。实验表明,该模型能够使用多个源,并改善了单源基线。

英文摘要:

In multi-source sequence-to-sequence tasks, the attention mechanism can be modeled in several ways. This topic has been thoroughly studied on recurrent architectures. In this paper, we extend the previous work to the encoder-decoder attention in the Transformer architecture. We propose four different input combination strategies for the encoder-decoder attention: serial, parallel, flat, and hierarchical. We evaluate our methods on tasks of multimodal translation and translation with multiple source languages. The experiments show that the models are able to use multiple sources and improve over single source baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑