arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.13362cs.CL
收藏

Tailor:一种基于属性的受控文本生成方法

Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation

National University of Singapore(新加坡国立大学);Alibaba Group(阿里巴巴集团)

Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, Jun Xie

摘要:

基于属性的受控文本生成(CTG)是指生成满足期望属性(例如,情感和主题)。现有的作品往往利用微调或诉诸额外的属性分类器,但遭受存储和推理时间的增加。为了解决这些问题,我们探索基于属性的CTG在一个基于属性的方式。简而言之,所提出的Tailor将每个属性表示为预先训练的连续向量(即,单属性提示),并引导生成固定的PLM开关到预先指定的属性。我们的实验发现,这些提示可以简单地连接作为一个整体,多属性CTG没有任何重新训练,但提出的问题,流畅性下降和位置敏感性。为此,Tailor提供了一个多属性提示掩码和一个重新索引的位置ID序列,以弥合训练(每个任务一个提示)和测试阶段(连接多个提示)之间的差距。为了进一步增强这种单属性提示组合,Tailor还引入了一个可训练的提示连接器,它可以与任何两个单属性提示连接到多属性文本生成。在GPT-2的训练参数为0.08%的情况下,对11个属性特定生成任务的实验表明,Tailor在单属性和多属性CTG上都有很好的性能.

英文摘要:

Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing works often utilize fine-tuning or resort to extra attribute classifiers, yet suffer from storage and inference time increases. To address these concerns, we explore attribute-based CTG in a prompt-based manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt) and guides the generation of a fixed PLM switch to a pre-specified attribute. We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity. To this end, Tailor provides a multi-attribute prompt mask and a re-indexing position-ids sequence to bridge the gap between the training (one prompt for each task) and testing stage (concatenating more than one prompt). To further enhance such single-attribute prompt combinations, Tailor also introduces a trainable prompt connector, which can be concatenated with any two single-attribute prompts to multi-attribute text generation. Experiments on 11 attribute-specific generation tasks demonstrate strong performances of Tailor on both single-attribute and multi-attribute CTG, with 0.08\% training parameters of a GPT-2.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑