FUDGE:使用未来鉴别器的受控文本生成
FUDGE: Controlled Text Generation With Future Discriminators
摘要:
我们提出了未来的鉴别器生成(FUDGE),一个灵活的和模块化的方法控制的文本生成。给定用于从兴趣分布生成文本的预先存在的模型G,FUDGE使得能够对期望的属性a(例如,形式)进行调节,同时仅需要访问G的输出logits。FUDGE学习对部分序列进行操作的属性预测器,并使用该预测器的输出来调整G的原始概率。我们表明,FUDGE模型的条款对应的贝叶斯分解的条件分布的G给定属性a。此外,FUDGE可以很容易地组成多个所需的属性的预测。我们在三个任务上评估FUDGE--诗歌中的对联完成,语言生成中的主题控制,以及机器翻译中的形式变化--并观察所有三个任务的收益。
英文摘要:
We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation. Given a pre-existing model G for generating text from a distribution of interest, FUDGE enables conditioning on a desired attribute a (for example, formality) while requiring access only to G's output logits. FUDGE learns an attribute predictor operating on a partial sequence, and uses this predictor's outputs to adjust G's original probabilities. We show that FUDGE models terms corresponding to a Bayesian decomposition of the conditional distribution of G given attribute a. Moreover, FUDGE can easily compose predictors for multiple desired attributes. We evaluate FUDGE on three tasks -- couplet completion in poetry, topic control in language generation, and formality change in machine translation -- and observe gains in all three tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。