Previous work on controllable text generation has explored the idea of control from the latent space, such as optimizing a representation with attribute-related classifiers or sampling a representation from relevant discrete samples. However, they are not effective enough in modeling both the latent space and the control, leaving controlled text with low quality and diversity. In this work, we propose a novel control framework using probability density estimation in the latent space. Our method utilizes an invertible transformation function, the Normalizing Flow, that maps the complex distributions in the latent space to simple Gaussian distributions in the prior space. Thus, we can perform sophisticated and flexible control in the prior space and feed the control effects back into the latent space owing to the one-one-mapping property of invertible transformations. Experiments on single-attribute controls and multi-attribute control reveal that our method outperforms several strong baselines on attribute relevance and text quality and achieves the SOTA. Further analysis of control strength adjustment demonstrates the flexibility of our control strategy.
Improved Paraphrase Generation via Controllable Latent Diffusion
National Key Laboratory for Novel Software Technology(计算机软件新技术国家重点实验室);Collaborative Innovation Center of Novel Software Technology and Industrialization(软件新技术与产业化协同创新中心);Nanjing University(南京大学)
MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学)
An Overview on Controllable Text Generation via Variational Auto-Encoders
University of Chinese Academy of Sciences(中国科学院大学);Noah’s Ark Lab, Huawei(华为诺亚方舟实验室);Huawei Technologies Ltd.(华为技术有限公司)
基于神经元的生成建模的最新进展重新点燃了人们对拥有能够与人类对话并能够理解自然语言的计算机系统的希望。深度神经架构的使用已经在许多上下文和任务中进行了大量探索,以满足各种用户需求。一方面,生成满足特定要求的文本内容是模型与不同人群无缝对话的优先事项。另一方面,潜变量模型(LVM),如变分自动编码器(VAE)作为最流行的生成模型的流派之一,旨在表征文本数据的分布模式。因此,他们天生就有能力学习值得探索的可控追求的整体文本特征。\noindent这篇综述介绍了现有的生成方案,与文本变分自动编码器相关的问题,并回顾了几个关于可控生成的应用程序,这些应用程序是这些通用配方的实例,\footnote{A detailed paper list is available at \url{https://github.com/ImKeTT/CTG-latentAEs}}以及相关的数据集,度量和讨论未来的研究。希望这篇综述能对变分自动编码器下可控语言生成的问题、流行方法和原始思想提供一个概述。
MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space
Tsinghua University(清华大学);ESTATES PTE LTD(6ESTATES PTE LTD);Beijing University of Posts and Telecommunications(北京邮电大学);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Institute for Artificial Intelligence, Tsinghua University(清华大学人工智能研究院);State Key Lab on Intelligent Technology and Systems, Tsinghua University(智能技术与系统国家重点实验室)
Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);The MOE Key Laboratory of Computational Linguistics, Peking University(北京大学计算语言学教育部重点实验室)
DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
University of Chinese Academy of Sciences(中国科学院大学);Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);ByteDance Inc.(字节跳动公司)