可控文本生成（Controllable Text Generation, CTG）是自然语言生成（natural language generation, NLG）领域的新兴方向，被认为是发展更能满足实际应用中特定约束的先进文本生成技术的关键。近年来，基于大规模预训练语言模型（pre-trained language models, PLMs）的方法，尤其是应用广泛的基于Transformer的PLMs，已成为NLG的新范式，能够生成更多样、更流畅的文本。然而，由于深度神经网络的可解释性有限，这类方法的可控性需要得到保障。为此，基于Transformer PLMs的可控文本生成已成为发展迅速但颇具挑战的新研究热点。近3-4年间涌现了多种多样的方法，针对需要不同类型控制约束的不同CTG任务展开研究。本文对该领域的常见任务、主要方法和评估方法进行了系统性的批判性综述。最后，我们讨论了该领域面临的挑战，并提出了多个有前景的未来方向。据我们所知，这是首篇从基于Transformer的PLMs视角总结前沿CTG技术的综述论文。我们希望它能帮助相关领域的研究者和从业者快速追踪学术与技术前沿，为他们呈现该领域的全貌并提供未来研究的路线图。
英文摘要：
Controllable Text Generation (CTG) is emerging area in the field of natural language generation (NLG). It is regarded as crucial for the development of advanced text generation technologies that better meet the specific constraints in practical applications. In recent years, methods using large-scale pre-trained language models (PLMs), in particular the widely used transformer-based PLMs, have become a new paradigm of NLG, allowing generation of more diverse and fluent text. However, due to the limited level of interpretability of deep neural networks, the controllability of these methods need to be guaranteed. To this end, controllable text generation using transformer-based PLMs has become a rapidly growing yet challenging new research hotspot. A diverse range of approaches have emerged in the recent 3-4 years, targeting different CTG tasks that require different types of controlled constraints. In this paper, we present a systematic critical review on the common tasks, main approaches, and evaluation methods in this area. Finally, we discuss the challenges that the field is facing, and put forward various promising future directions. To the best of our knowledge, this is the first survey paper to summarize the state-of-the-art CTG techniques from the perspective of Transformer-based PLMs. We hope it can help researchers and practitioners in the related fields to quickly track the academic and technological frontier, providing them with a landscape of the area and a roadmap for future research.
Controllable Text Generation with Residual Memory Transformer
School of Computer Science & Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Nagoya University（名古屋大学）；Insight Edge, Inc.（Insight Edge 公司）；The Asahi Shimbun Company（朝日新闻社）；Tokyo Institute of Technology（东京工业大学）；Tohoku University（东北大学）；RIKEN AIP（理化学研究所革新智能综合研究中心）
Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；The MOE Key Laboratory of Computational Linguistics, Peking University（北京大学计算语言学教育部重点实验室）
