Neural controllable text generation is an important area gaining attention due to its plethora of applications. Although there is a large body of prior work in controllable text generation, there is no unifying theme. In this work, we provide a new schema of the pipeline of the generation process by classifying it into five modules. The control of attributes in the generation process requires modification of these modules. We present an overview of different techniques used to perform the modulation of these modules. We also provide an analysis on the advantages and disadvantages of these techniques. We further pave ways to develop new architectures based on the combination of the modules described in this paper.
Towards Attribute-Entangled Controllable Text Generation: A Pilot Study of Blessing Generation
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of Intelligent Systems Engineering, Sun-Yat Sen University(中山大学智能系统工程学院);Peng Cheng Laboratory(鹏城实验室)
An Overview on Controllable Text Generation via Variational Auto-Encoders
University of Chinese Academy of Sciences(中国科学院大学);Noah’s Ark Lab, Huawei(华为诺亚方舟实验室);Huawei Technologies Ltd.(华为技术有限公司)
基于神经元的生成建模的最新进展重新点燃了人们对拥有能够与人类对话并能够理解自然语言的计算机系统的希望。深度神经架构的使用已经在许多上下文和任务中进行了大量探索,以满足各种用户需求。一方面,生成满足特定要求的文本内容是模型与不同人群无缝对话的优先事项。另一方面,潜变量模型(LVM),如变分自动编码器(VAE)作为最流行的生成模型的流派之一,旨在表征文本数据的分布模式。因此,他们天生就有能力学习值得探索的可控追求的整体文本特征。\noindent这篇综述介绍了现有的生成方案,与文本变分自动编码器相关的问题,并回顾了几个关于可控生成的应用程序,这些应用程序是这些通用配方的实例,\footnote{A detailed paper list is available at \url{https://github.com/ImKeTT/CTG-latentAEs}}以及相关的数据集,度量和讨论未来的研究。希望这篇综述能对变分自动编码器下可控语言生成的问题、流行方法和原始思想提供一个概述。
Length-controllable Abstractive Summarization by Guiding with Summary Prototype
NTT Media Intelligence Laboratories(日本电信电话公司媒体智能实验室);NTT Corporation(日本电信电话公司);Nara Institute of Science Technology(奈良先端科学技术大学院大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心)
MReD: A Meta-Review Dataset for Structure-Controllable Text Generation
DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);National University of Singapore(新加坡国立大学);Singapore University of Technology and Design(新加坡科技设计大学);Nanyang Technological University(南洋理工大学)
Improved Paraphrase Generation via Controllable Latent Diffusion
National Key Laboratory for Novel Software Technology(计算机软件新技术国家重点实验室);Collaborative Innovation Center of Novel Software Technology and Industrialization(软件新技术与产业化协同创新中心);Nanjing University(南京大学)
PG-CE: A Progressive Generation Dataset with Constraint Enhancement for Controllable Text Generation
Minzu University of China. National Language Resource Monitoring \& Research Center Minority Languages Branch \& Institute of National Security, Minzu University of China Beijing, China zhuangyan\;Minzu University of China. National Language Resource Monitoring Research Center Minority Languages Branch \& Institute of National Security, Minzu University of China Beijing, China
MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学)
Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Beijing University of Posts and Telecommunications(北京邮电大学);Nanyang Technological University(南洋理工大学)
Controllable Text Generation with Residual Memory Transformer
School of Computer Science & Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系)
EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models
Department of Electrical and Computer Engineering, Seoul National University(电子与计算机工程系,首尔国立大学);AIIS, ASRI, INMC, ISRC, and IPAI, Seoul National University(人工智能研究所、先进系统研究室、智能网络中心、信息科学研究中心和人工智能计划,首尔国立大学)
Mix and Match: Learning-free Controllable Text Generation using Energy Language Models
University of California San Diego(加州大学圣迭戈分校);Toyota Technological Institute at Chicago (TTIC)(芝加哥丰田技术研究院(TTIC))
控制文本生成的最新工作要么需要基于属性的微调的基础语言模型(LM),或限制了参数化的属性映射,以兼容的基础自回归LM。在这项工作中,我们提出了Mix and Match LM,这是一种基于全局分数的可控文本生成方法,它结合了任意预训练的黑盒模型,用于在生成的文本中实现所需的属性,而不涉及任何关于黑盒模型的微调或结构假设。我们将可控生成的任务解释为从基于能量的模型中抽取样本,该模型的能量值是来自黑盒模型的分数的线性组合,黑盒模型分别负责流畅性、控制属性和对任何条件上下文的忠诚度。我们使用Metropolis-Hastings抽样方案从这个基于能量的模型中使用双向上下文和全局属性特征进行抽样。我们验证了我们的方法在各种控制生成和基于风格的文本修订任务的有效性,优于最近提出的方法,涉及额外的训练,微调,或限制性的假设模型的形式。
Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts
Paul G. Allen School of Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院);Allen Institute for AI(艾伦人工智能研究所);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)
MaRCo 结合可控生成与专家乘积,利用无毒和有毒语言模型似然识别并替换候选词,在微妙毒性去毒任务上自动指标优于基线且人类评估偏好达 2.1 倍。
Reinforcement Learning with Token-level Feedback for Controllable Text Generation
Ping An Property & Casualty Insurance company of China(中国平安财产保险股份有限公司);The Chinese University of Hong Kong(香港中文大学);Huazhong University of Science and Technology(华中科技大学);Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL)(华中科技大学与平安财产保险研究联合实验室)
An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation
Meituan(美团);Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?
School of New Media and Communication, Tianjin University(天津大学新媒体与传播学院);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部)
Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation
University of Science and Technology of China(中国科学技术大学);Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);Chinese Academy of Sciences(中国科学院);BOSS Zhipin Career Science Lab(BOSS智联招聘科研实验室)