Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing works often utilize fine-tuning or resort to extra attribute classifiers, yet suffer from storage and inference time increases. To address these concerns, we explore attribute-based CTG in a prompt-based manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt) and guides the generation of a fixed PLM switch to a pre-specified attribute. We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity. To this end, Tailor provides a multi-attribute prompt mask and a re-indexing position-ids sequence to bridge the gap between the training (one prompt for each task) and testing stage (concatenating more than one prompt). To further enhance such single-attribute prompt combinations, Tailor also introduces a trainable prompt connector, which can be concatenated with any two single-attribute prompts to multi-attribute text generation. Experiments on 11 attribute-specific generation tasks demonstrate strong performances of Tailor on both single-attribute and multi-attribute CTG, with 0.08\% training parameters of a GPT-2.
The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation
The Chinese University of Hong Kong(香港中文大学);Harbin Institute of Technology(哈尔滨工业大学);Monash University Malaysia(莫纳什大学马来西亚分校);Peng Cheng Laboratory(鹏城实验室)
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Georgia Institute of Technology(佐治亚理工学院);University of Washington(华盛顿大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Google Research(谷歌研究院)
Controllable Image Captioning with Prompt-Conditioned Scene Rewards
POSTECH(浦项科技大学);Graduate School of Artificial Intelligence, POSTECH(浦项科技大学人工智能研究生院);Department of Computer Science and Engineering, POSTECH(浦项科技大学计算机科学与工程系)
Divide & Bind Your Attention for Improved Generative Semantic Nursing
Bosch Center for Artificial Intelligence(博世人工智能中心);University of Mannheim(曼海姆大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);University of Tübingen(蒂宾根大学)