InsNet：一种高效、灵活、高性能的基于插入的文本生成模型
InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model
摘要：
我们提出了InsNet，这是一个基于插入的文本生成器，具有高效的训练和灵活的解码（并行或顺序）。与大多数现有的基于插入的文本生成工作不同，这些工作需要在每次插入操作之后重新编码上下文，因此训练效率低下，InsNet通过引入新颖的面向插入的位置编码和轻量级槽表示策略来实现计算共享，从而在训练期间仅需要对整个序列进行一次上下文编码。此外，我们提出了一个算法InsNet-Dinic，以更好地确定插入操作的并行化，提供了一个可控的并行和顺序解码之间的切换，使其灵活地处理更多的并行化任务，如机器翻译与有效的解码，或较少的并行化任务，如开放域文本生成，以保证高质量的输出。在两个词汇约束文本生成数据集和三个机器翻译数据集上的实验证明了InsNet在训练速度、推理效率和生成质量方面优于以前基于插入的方法。
英文摘要：
We propose InsNet, an expressive insertion-based text generator with efficient training and flexible decoding (parallel or sequential). Unlike most existing insertion-based text generation works that require re-encoding of the context after each insertion operation and thus are inefficient to train, InsNet only requires one pass of context encoding for the entire sequence during training by introducing a novel insertion-oriented position encoding and a light-weighted slot representation strategy to enable computation sharing. Furthermore, we propose an algorithm InsNet-Dinic to better determine the parallelization of insertion operations that provides a controllable switch between parallel and sequential decoding, making it flexible to handle more parallelizable tasks such as machine translation with efficient decoding, or less parallelizable tasks such as open-domain text generation to guarantee high-quality outputs. Experiments on two lexically constrained text generation datasets and three machine translation datasets demonstrate InsNet's advantages over previous insertion-based methods in terms of training speed, inference efficiency, and generation quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
