基于BART的词汇约束文本生成的并行优化
Parallel Refinements for Lexically Constrained Text Generation with BART
摘要：
词汇约束文本生成的目的是通过将一些预先指定的关键字合并到输出中来控制生成的文本。以前的工作通过控制解码过程或迭代地细化候选输出来向输出中注入词汇约束，这往往会产生通用或不符合语法的句子，并且具有很高的计算复杂性。为了解决这些挑战，我们提出了约束BART（CBART）词汇约束的文本生成。CBART利用预训练模型BART，通过将该任务分解为两个子任务，将部分生成负担从解码器转移到编码器，从而提高句子质量。具体来说，我们通过在编码器上添加标记级分类器来扩展BART，旨在指导解码器在哪里替换和插入。在编码器的指导下，解码器通过在特定位置之前插入令牌并以低置信度重新预测令牌，在一个步骤中细化输入的多个令牌。为了进一步减少推理延迟，解码器并行预测所有令牌。在One-Billion-Word和Yelp上的实验结果表明，CBART能够生成高质量、多样性的可信文本，同时显著加快推理速度。
英文摘要：
Lexically constrained text generation aims to control the generated text by incorporating some pre-specified keywords into the output. Previous work injects lexical constraints into the output by controlling the decoding process or refining the candidate output iteratively, which tends to generate generic or ungrammatical sentences, and has high computational complexity. To address these challenges, we propose Constrained BART (CBART) for lexically constrained text generation. CBART leverages the pre-trained model BART and transfers part of the generation burden from the decoder to the encoder by decomposing this task into two sub-tasks, thereby improving the sentence quality. Concretely, we extend BART by adding a token-level classifier over the encoder, aiming at instructing the decoder where to replace and insert. Guided by the encoder, the decoder refines multiple tokens of the input in one step by inserting tokens before specific positions and re-predicting tokens with low confidence. To further reduce the inference latency, the decoder predicts all tokens in parallel. Experiment results on One-Billion-Word and Yelp show that CBART can generate plausible text with high quality and diversity while significantly accelerating inference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
