插入-缺失Transformer
Insertion-Deletion Transformer
摘要：
我们提出了插入删除Transformer，一种新的基于transformer的神经结构和训练方法的序列生成。该模型由迭代执行的两个阶段组成，1）插入阶段和2）删除阶段。插入阶段参数化当前输出假设上的插入分布，而删除阶段参数化当前输出假设上的删除分布。训练方法是一种原理性的简单算法，其中删除模型直接从插入模型输出中获得其信号。我们证明了我们的插入-删除Transformer在合成翻译任务上的有效性，与仅插入模型相比，BLEU得分得到了显着提高。
英文摘要：
We propose the Insertion-Deletion Transformer, a novel transformer-based neural architecture and training method for sequence generation. The model consists of two phases that are executed iteratively, 1) an insertion phase and 2) a deletion phase. The insertion phase parameterizes a distribution of insertions on the current output hypothesis, while the deletion phase parameterizes a distribution of deletions over the current output hypothesis. The training method is a principled and simple algorithm, where the deletion model obtains its signal directly on-policy from the insertion model output. We demonstrate the effectiveness of our Insertion-Deletion Transformer on synthetic translation tasks, obtaining significant BLEU score improvement over an insertion-only model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
