Prompt tuning is a new, efficient NLP transfer learning paradigm that adds a task-specific prompt in each input instance during the model training stage. It freezes the pre-trained language model and only optimizes a few task-specific prompts. In this paper, we propose a conditional prompt generation method to generate prompts for each input instance, referred to as the Instance-Dependent Prompt Generation (IDPG). Unlike traditional prompt tuning methods that use a fixed prompt, IDPG introduces a lightweight and trainable component to generate prompts based on each input sentence. Extensive experiments on ten natural language understanding (NLU) tasks show that the proposed strategy consistently outperforms various prompt tuning baselines and is on par with other efficient transfer learning methods such as Compacter while tuning far fewer model parameters.
Instance-aware Prompt Learning for Language Understanding and Generation
National Laboratory of Pattern Recognition, Institute of Automation, CAS（中国科学院自动化研究所模式识别国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Alibaba Cloud Computing（阿里巴巴云计算）；Shanghai Jiao Tong University（上海交通大学）；College of Software, Zhejiang University（浙江大学软件学院）
All You Need is One: Capsule Prompt Tuning with a Single Vector
University of Missouri-Kansas City（密苏里大学堪萨斯城分校）；U.S. Naval Research Laboratory（美国海军研究实验室）；University of North Texas（北卡罗来纳州立大学）；Lamar University（拉马尔大学）；ByteDance（字节跳动）；Case Western Reserve University（凯斯西储大学）；University of California, Davis（加州大学戴维斯分校）；Rochester Institute of Technology（罗切斯特理工学院）；Meta AI
GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning
Université d’Orléans（奥尔良大学）；INSA CVL（中央卢瓦尔河谷国立应用科学学院）；CY Cergy Paris University（塞尔吉-蓬图瓦兹大学）；ENSEA（法国高等电子与数字学院）；CNRS（法国国家科学研究中心）；Université de Technologie de Troyes（特鲁瓦技术大学）
Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis
Harvard University（哈佛大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Brown University（布朗大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Georgia Institute of Technology（佐治亚理工学院）
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）
Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；School of Information and Control Engineering, China University of Mining and Technology（中国矿业大学信息与控制工程学院）
DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；IBM Research AI（IBM研究院人工智能）；MIT（麻省理工学院）；Technion - Israel Institute of Technology（以色列理工学院）；Allen Institute for AI（艾伦人工智能研究所）
MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion
Department of Computing, The Hong Kong Polytechnic University（计算系，香港理工大学）；Research Institute of Multiple Agents and Embodied Intelligence, Pengcheng Laboratory（多智能体与具身智能研究院，鹏城实验室）
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Université de Montréal（蒙特利尔大学）；School of Information, Renmin University of China（中国人民大学信息学院）
