arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1608.05554cs.CL
收藏

学习如何启动序列到序列架构

Learning to Start for Sequence to Sequence Architecture

Harbin Institute of Technology(哈尔滨工业大学);Tencent Corporation(腾讯公司)

Qingfu Zhu, Weinan Zhang, Lianqiang Zhou, Ting Liu

摘要:

序列到序列架构被广泛用于响应生成和神经机器翻译中,以对两个句子之间的潜在关系建模。它通常由两部分组成:一个从源句子中读取的编码器和一个根据编码器的输出和最后生成的单词逐词生成目标句子的解码器。然而,它面临的冷启动问题时,产生的第一个词,因为没有前面的词参考。现有的工作主要是使用一个特殊的开始符号</s>来产生第一个词。这些工作的一个明显缺点是,单词和开始符号之间没有可学习的关系。此外,当第一个字不正确地生成时,可能导致解码的错误累积。在本文中,我们提出了一种新的方法来学习生成序列到序列架构中的第一个字,而不是使用开始符号。在短文本会话的响应生成任务上的实验结果表明,该方法在自动和人工评估方面都优于现有的方法。

英文摘要:

The sequence to sequence architecture is widely used in the response generation and neural machine translation to model the potential relationship between two sentences. It typically consists of two parts: an encoder that reads from the source sentence and a decoder that generates the target sentence word by word according to the encoder's output and the last generated word. However, it faces to the cold start problem when generating the first word as there is no previous word to refer. Existing work mainly use a special start symbol </s>to generate the first word. An obvious drawback of these work is that there is not a learnable relationship between words and the start symbol. Furthermore, it may lead to the error accumulation for decoding when the first word is incorrectly generated. In this paper, we proposed a novel approach to learning to generate the first word in the sequence to sequence architecture rather than using the start symbol. Experimental results on the task of response generation of short text conversation show that the proposed approach outperforms the state-of-the-art approach in both of the automatic and manual evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑