Improving the controllability, portability, and inference speed of diffusion language models (DLMs) is a key challenge in natural language generation. While recent research has shown significant success in complex text generation with language models, the memory and computational power are still very demanding and fall short of expectations, which naturally results in low portability and instability for the models. To mitigate these issues, numerous well-established methods were proposed for neural network quantization. To further enhance their portability of independent deployment as well as improve their stability evaluated by language perplexity, we propose a novel approach called the Quantized Embedding Controllable Diffusion Language Model (QE-CDLM). QE-CDLM builds upon the recent successful controllable DLMs by remodeling the task-specific embedding space via quantization. This leads to a gradient-based controller for the generation tasks, and more stable intermediate latent variables are obtained, which naturally brings in an accelerated convergence as well as better controllability. Additionally, the adaption fine-tuning method is employed to reduce tunable weights. Experimental results on five challenging fine-grained control tasks demonstrate that QE-CDLM compares favorably to existing methods in terms of quality and feasibility, achieving better perplexity and lightweight fine-tuning.
Length-controllable Abstractive Summarization by Guiding with Summary Prototype
NTT Media Intelligence Laboratories(日本电信电话公司媒体智能实验室);NTT Corporation(日本电信电话公司);Nara Institute of Science Technology(奈良先端科学技术大学院大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心)
Quark: Controllable Text Generation with Reinforced Unlearning
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Paul G. Allen School of Computer Science, University of Washington(华盛顿大学保罗·G·艾伦计算机科学学院)
大规模的语言模型经常学习与用户期望不一致的行为。生成的文本可能包含攻击性或有毒语言,包含大量重复,或者与用户期望的不同情绪。我们考虑的任务是通过微调语言模型的信号不做什么来消除这些错位。我们引入了量化奖励约束(Quark),这是一种优化奖励函数的算法,该函数量化了(不)想要的属性,同时不会偏离原始模型太远。Quark在(i)收集具有当前语言模型的样本,(ii)基于奖励将它们分类为分位数,每个分位数由预先添加到语言模型的输入的奖励令牌标识,以及(iii)在来自每个分位数的以其奖励令牌为条件的样本上使用标准语言建模损失,同时通过KL发散惩罚保持在原始语言模型附近。通过在生成时以高奖励令牌为条件,该模型生成的文本显示出较少的不需要的属性。对于消除毒性,消极情绪和重复,我们的实验表明Quark优于强基线和最先进的强化学习方法,如PPO(Schulman et al. 2017),同时仅依赖于标准语言建模原语。
A Survey of Quantized Graph Representation Learning: Connecting Graph Structures with Large Language Models
National University of Singapore(新加坡国立大学);Xi’an Jiaotong University(西安交通大学);University of Southern Queensland(南方昆士兰大学);Nanyang Technological University(南洋理工大学)