自动推断生成顺序的基于插入的译码
Insertion-based Decoding with automatically Inferred Generation Order
摘要:
传统的神经自回归解码通常假定固定的从左到右的生成顺序,这可能是次优的。在这项工作中,我们提出了一种新的解码算法- InDIGO -它支持灵活的序列生成任意顺序通过插入操作。我们扩展了Transformer,一个最先进的序列生成模型,以有效地实现所提出的方法,使其能够与预定义的生成顺序或自适应订单从波束搜索得到的训练。在词序恢复、机器翻译、图像字幕和代码生成等四个实际任务上的实验表明,该算法可以生成任意顺序的序列,同时与传统的从左到右生成算法相比,具有相当甚至更好的性能。生成的序列表明,InDIGO采用自适应生成顺序的基础上输入信息。
英文摘要:
Conventional neural autoregressive decoding commonly assumes a fixed left-to-right generation order, which may be sub-optimal. In this work, we propose a novel decoding algorithm -- InDIGO -- which supports flexible sequence generation in arbitrary orders through insertion operations. We extend Transformer, a state-of-the-art sequence generation model, to efficiently implement the proposed approach, enabling it to be trained with either a pre-defined generation order or adaptive orders obtained from beam-search. Experiments on four real-world tasks, including word order recovery, machine translation, image caption and code generation, demonstrate that our algorithm can generate sequences following arbitrary orders, while achieving competitive or even better performance compared to the conventional left-to-right generation. The generated sequences show that InDIGO adopts adaptive generation orders based on input information.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。