arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.04160cs.CL
收藏

利用提示技术发挥即插即用控制器的作用

Harnessing the Plug-and-Play Controller by Prompting

Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Zhongguancun Laboratory(中关村实验室)

Hao Wang, Lei Sha

AI总结:

本文提出一种利用提示技术引导即插即用控制器(PPC)的新方法,通过动态调整提示约束输出空间,并创新性地引入RLDAF微调方法,在SST2数据集上提升了文本生成的流畅性和属性一致性。

摘要:

可控文本生成是自然语言生成(NLG)领域中一个不断发展的研究方向,其重点是在实际应用中生成满足特定约束的文本。以往的方法,如即插即用控制器(PPC),旨在以灵活的方式引导生成文本的属性。然而,这些方法往往损害了语言模型解码过程的完整性,导致文本生成不够流畅。另外,其他技术利用多个属性提示来使生成文本与所需属性对齐,但这种方法需要为每个属性设计提示,并且依赖于语言模型的规模。本文提出了一种利用预训练语言模型(PLM)进行文本生成中灵活属性控制的新方法。所提出的方法旨在通过使用PPC引导生成过程来增强生成文本的流畅性。其关键思想是通过修改提示来动态调整生成文本的分布,从而有效约束语言模型的输出空间并影响所需属性。为了实现PLM与PPC之间的平滑协作,我们的工作创新性地提出了一种新的模型微调方法:基于动态调整反馈的强化学习(RLDAF)。该微调过程根据PPC控制过程中采取的生成动作,调整语言模型参数的一小部分子集。由此产生的PLM与PPC之间的和谐协作,使得推理过程中的文本生成流畅性得到改善。我们在SST2数据集上进行了大量实验,所提出的方法在文本流畅性和属性一致性等多个评估指标上优于以往的方法。

英文摘要:

Controllable text generation is a growing field within natural language generation (NLG) that focuses on producing text that meets specific constraints in real-world applications. Previous approaches, such as plug-and-play controllers (PPCs), aimed to steer the properties of generated text in a flexible manner. However, these methods often compromised the integrity of the language model's decoding process, resulting in less smooth text generation. Alternatively, other techniques utilized multiple attribute prompts to align the generated text with desired attributes, but this approach required prompt design for each attribute and was dependent on the size of the language model. This paper introduces a novel method for flexible attribute control in text generation using pre-trained language models (PLMs). The proposed approach aims to enhance the fluency of generated text by guiding the generation process with PPCs. The key idea is to dynamically adjust the distribution of generated text by modifying prompts, effectively constraining the output space of the language model and influencing the desired attribute. To enable smooth cooperation between the PLM and the PPC, our work innovatively proposes a new model fine-tuning method: Reinforcement Learning with Dynamic Adjust Feedback (RLDAF).This fine-tuning process adapts a small subset of the language model's parameters based on the generating actions taken during the PPC control process. The resulting harmonious collaboration between the PLM and PPC leads to improved smoothness in text generation during inference. Extensive experiments were conducted on the SST2 dataset, and the proposed method outperformed previous approaches in various evaluation metrics, including text fluency and attribute consistency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑