arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1810.13409cs.CL
收藏

您可能不需要注意

You May Not Need Attention

Paul G. Allen School of Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院);Allen Institute for Artificial Intelligence(艾伦人工智能研究所)

Ofir Press, Noah A. Smith

摘要:

在NMT中,没有注意力,没有单独的编码和解码,我们能走多远?为了回答这个问题,我们引入了一个循环神经翻译模型,它不使用注意力,也没有单独的编码器和解码器。我们的即时翻译模型是低延迟的,在读取第一个源令牌时立即写入目标令牌,并在解码期间使用恒定内存。它的表现与Bahdanau et al.(2014)的标准注意力模型相当,并且在长句上表现更好。

英文摘要:

In NMT, how far can we get without attention and without separate encoding and decoding? To answer that question, we introduce a recurrent neural translation model that does not use attention and does not have a separate encoder and decoder. Our eager translation model is low-latency, writing target tokens as soon as it reads the first source token, and uses constant memory during decoding. It performs on par with the standard attention-based model of Bahdanau et al. (2014), and better on long sentences.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑