arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.06930cs.CL
收藏

LiFi:基于细粒度控制码的轻量级受控文本生成

LiFi: Lightweight Controlled Text Generation with Fine-Grained Control Codes

Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Tencent AI Lab(腾讯AI Lab)

Chufan Shi, Deng Cai, Yujiu Yang

AI总结:

提出 LIFI 方法,利用连续、相对且非互斥的细粒度控制码结合 adapters,在少量标注和大量未标注数据上实现轻量级受控文本生成,在情感、主题控制和风格化小说写作任务上显著优于基线。

摘要:

在快速发展的文本生成领域,对更精确控制机制的需求日益显著。为满足这一需求,我们提出了一种新颖的方法 LIFI,它为受控文本生成提供了一种具有细粒度控制的轻量级方法。以往研究训练预训练语言模型遵循离散、分类且互斥的控制码,与之不同的是,LIFI 在连续、相对且非互斥的控制码指导下学习受控文本生成。这些细粒度控制码自动从属性分类器中推导得出,该分类器最初用少量标注数据训练,随后被用于标注大量未标注数据,从而获取更广泛的监督信号。此外,为实现高效控制,我们将细粒度控制码与 adapters 结合,这是一种参数和计算高效地引导预训练语言模型的方法。我们在两个常规任务——情感控制和主题控制——以及一个新提出的任务——风格化小说写作上评估了 LIFI。全面的实验结果验证了我们提出方法的有效性,表明相比现有基线有显著的性能提升。

英文摘要:

In the rapidly evolving field of text generation, the demand for more precise control mechanisms has become increasingly apparent. To address this need, we present a novel methodology, LIFI, which offers a lightweight approach with fine-grained control for controlled text generation. Unlike previous studies that train pre-trained language models to follow discrete, categorical, and exclusive control codes, LIFI learns controlled text generation under the guidance of continuous, relative, and nonexclusive control codes. These fine-grained codes are automatically derived from an attribute classifier, initially trained with a small amount of labeled data and subsequently employed to label abundant unlabeled data, thus garnering more extensive supervision signals. Moreover, to achieve efficient control, we incorporate the fine-grained control codes with adapters, a parameter- and compute-efficient way to steer a pre-trained language model. We evaluate LIFI on two conventional tasks -- sentiment control and topic control -- and one newly proposed task -- stylistic novel writing. Comprehensive experimental results validate the effectiveness of our proposed methods, demonstrating substantial performance improvements over existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑