Text generation tasks, including translation, summarization, language models, and etc. see rapid growth during recent years. Despite the remarkable achievements, the repetition problem has been observed in nearly all text generation models undermining the generation performance extensively. To solve the repetition problem, many methods have been proposed, but there is no existing theoretical analysis to show why this problem happens and how it is resolved. In this paper, we propose a new framework for theoretical analysis for the repetition problem. We first define the Average Repetition Probability (ARP) to characterize the repetition problem quantitatively. Then, we conduct an extensive analysis of the Markov generation model and derive several upper bounds of the average repetition probability with intuitive understanding. We show that most of the existing methods are essentially minimizing the upper bounds explicitly or implicitly. Grounded on our theory, we show that the repetition problem is, unfortunately, caused by the traits of our language itself. One major reason is attributed to the fact that there exist too many words predicting the same word as the subsequent word with high probability. Consequently, it is easy to go back to that word and form repetitions and we dub it as the high inflow problem. Furthermore, we derive a concentration bound of the average repetition probability for a general generation model. Finally, based on the theoretical upper bounds, we propose a novel rebalanced encoding approach to alleviate the high inflow problem. The experimental results show that our theoretical framework is applicable in general generation models and our proposed rebalanced encoding approach alleviates the repetition problem significantly. The source code of this paper can be obtained from https://github.com/fuzihaofzh/repetition-problem-nlg.
Instituto de Telecomunicações(电信研究所);Priberam Labs(Priberam实验室);Institute of Systems and Robotics(系统与机器人研究所);LUMLIS (Lisbon ELLIS Unit), Instituto Superior Técnico(里斯本ELLIS单位LUMLIS,高等技术学院);Unbabel(Unbabel公司);Lisbon, Portugal(葡萄牙里斯本)
Quark: Controllable Text Generation with Reinforced Unlearning
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Paul G. Allen School of Computer Science, University of Washington(华盛顿大学保罗·G·艾伦计算机科学学院)
大规模的语言模型经常学习与用户期望不一致的行为。生成的文本可能包含攻击性或有毒语言,包含大量重复,或者与用户期望的不同情绪。我们考虑的任务是通过微调语言模型的信号不做什么来消除这些错位。我们引入了量化奖励约束(Quark),这是一种优化奖励函数的算法,该函数量化了(不)想要的属性,同时不会偏离原始模型太远。Quark在(i)收集具有当前语言模型的样本,(ii)基于奖励将它们分类为分位数,每个分位数由预先添加到语言模型的输入的奖励令牌标识,以及(iii)在来自每个分位数的以其奖励令牌为条件的样本上使用标准语言建模损失,同时通过KL发散惩罚保持在原始语言模型附近。通过在生成时以高奖励令牌为条件,该模型生成的文本显示出较少的不需要的属性。对于消除毒性,消极情绪和重复,我们的实验表明Quark优于强基线和最先进的强化学习方法,如PPO(Schulman et al. 2017),同时仅依赖于标准语言建模原语。
On-the-Fly Attention Modulation for Neural Generation
Mila / McGill University(米拉实验室/麦吉尔大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Paul G. Allen School of CSE, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院);Stanford University(斯坦福大学)
Text Distance from Nested and Hierarchical Repetitions: A Compression-Based Perspective
Department of Systems Science, Faculty of Arts and Sciences, Beijing Normal University(北京师范大学文理学院系统科学系);International Academic Center of Complex Systems, Beijing Normal University(北京师范大学复杂系统国际学术中心);School of Systems Science, Beijing Normal University(北京师范大学系统科学学院);College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件学院)
Uncovering Uncontrolled Repetition through Residual Stream Dynamics
Beijing University of Posts and Telecommunications(北京邮电大学);JIUTIAN Research(中移九天);Nanyang Technological University(南洋理工大学);Chongqing University of Posts and Telecommunications(重庆邮电大学)