arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.08307cs.CL
收藏

通过潜在空间中的概率密度估计实现可控文本生成

Controllable Text Generation via Probability Density Estimation in the Latent Space

Harbin Institute of Technology(哈尔滨工业大学);Peng Cheng Laboratory(鹏城实验室)

Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, Weihong Zhong, Bing Qin

AI总结:

本文提出一种基于潜在空间概率密度估计的新型可控文本生成框架,利用 Normalizing Flow 将复杂分布映射为高斯分布以实现灵活控制,在单/多属性控制任务上达到 SOTA。

摘要:

先前关于可控文本生成的工作探索了从潜在空间进行控制的想法,例如使用与属性相关的分类器优化表示,或从相关的离散样本中采样表示。然而,它们在建模潜在空间和控制方面不够有效,导致受控文本的质量和多样性较低。在这项工作中,我们提出了一种在潜在空间中使用概率密度估计的新型控制框架。我们的方法利用一个可逆转换函数 Normalizing Flow,将潜在空间中的复杂分布映射到先验空间中的简单高斯分布。因此,由于可逆变换的一一映射特性,我们可以在先验空间中执行复杂且灵活的控制,并将控制效果反馈到潜在空间中。在单属性控制和多属性控制上的实验表明,我们的方法在属性相关性和文本质量上优于几个强基线,并达到了 SOTA。对控制强度调整的进一步分析证明了我们控制策略的灵活性。

英文摘要:

Previous work on controllable text generation has explored the idea of control from the latent space, such as optimizing a representation with attribute-related classifiers or sampling a representation from relevant discrete samples. However, they are not effective enough in modeling both the latent space and the control, leaving controlled text with low quality and diversity. In this work, we propose a novel control framework using probability density estimation in the latent space. Our method utilizes an invertible transformation function, the Normalizing Flow, that maps the complex distributions in the latent space to simple Gaussian distributions in the prior space. Thus, we can perform sophisticated and flexible control in the prior space and feed the control effects back into the latent space owing to the one-one-mapping property of invertible transformations. Experiments on single-attribute controls and multi-attribute control reveal that our method outperforms several strong baselines on attribute relevance and text quality and achieves the SOTA. Further analysis of control strength adjustment demonstrates the flexibility of our control strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑