All You Need is One: Capsule Prompt Tuning with a Single Vector
University of Missouri-Kansas City（密苏里大学堪萨斯城分校）；U.S. Naval Research Laboratory（美国海军研究实验室）；University of North Texas（北卡罗来纳州立大学）；Lamar University（拉马尔大学）；ByteDance（字节跳动）；Case Western Reserve University（凯斯西储大学）；University of California, Davis（加州大学戴维斯分校）；Rochester Institute of Technology（罗切斯特理工学院）；Meta AI
Yiyang Liu, James C. Liang, Heng Fan, Wenhao Yang, Yiming Cui, Xiaotian Han, Lifu Huang, Dongfang Liu, Qifan Wang, Cheng Han
Prompt-based learning has emerged as a parameter-efficient finetuning (PEFT) approach to facilitate Large Language Model (LLM) adaptation to downstream tasks by conditioning generation with task-aware guidance. Despite its successes, current prompt-based learning methods heavily rely on laborious grid searching for optimal prompt length and typically require considerable number of prompts, introducing additional computational burden. Worse yet, our pioneer findings indicate that the task-aware prompt design is inherently limited by its absence of instance-aware information, leading to a subtle attention interplay with the input sequence. In contrast, simply incorporating instance-aware information as a part of the guidance can enhance the prompt-tuned model performance without additional fine-tuning. Moreover, we find an interesting phenomenon, namely "attention anchor", that incorporating instance-aware tokens at the earliest position of the sequence can successfully preserve strong attention to critical structural information and exhibit more active attention interaction with all input tokens. In light of our observation, we introduce Capsule Prompt-Tuning (CaPT), an efficient and effective solution that leverages off-the-shelf, informative instance semantics into prompt-based learning. Our approach innovatively integrates both instance-aware and task-aware information in a nearly parameter-free manner (i.e., one single capsule prompt). Empirical results demonstrate that our method can exhibit superior performance across various language tasks (e.g., 84.03\% average accuracy on T5-Large), serving as an "attention anchor," while enjoying high parameter efficiency (e.g., 0.003\% of model parameters on Llama3.2-1B).
R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
Tianjin University（天津大学）；Tencent Hunyuan（腾讯文元）；Peking University（北京大学）；Fudan University（复旦大学）；National Engineering Research Center of Educational Big Data and the Faculty of Artificial Intelligence in Education, Central China Normal University（教育大数据国家工程研究中心和教育人工智能学院，中央财经大学）
Prompt Tuning for Discriminative Pre-trained Language Models
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；National University of Singapore（新加坡国立大学）；Tencent（腾讯）
PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning
School of Information Science & Technology, Pennsylvania State University（宾夕法尼亚州立大学信息科学与技术学院）；Department of Computer Science, Stony Brook University（石溪大学计算机科学系）
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））；University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校（UIUC））；Imperial College London（伦敦帝国学院）
Tsinghua University（清华大学）；Zhongguancun Laboratory（中关村实验室）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
PromptRad: Knowledge-Enhanced Multi-Label Prompt-Tuning for Low-Resource Radiology Report Labeling
Department of Artificial Intelligence and AI Research Center, Chang Gung University（人工智能系及AI研究中心，长庚大学）；Department of Radiology, Sijhih Cathay General Hospital（放射科，西吉医院）；Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital（医学影像与介入科，长庚纪念医院）；Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital（创伤与急诊外科，长庚纪念医院）；Department of Computer Science, National Tsing Hua University（计算机科学系，国立清华大学）
College of Computer Science and Software Engineering, Shenzhen University, China（深圳大学计算机科学与软件工程学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院）；Guangdong Lab of AI and Digital Economy (SZ), China（广东人工智能与数字经济实验室（深圳））；Hong Kong University of Science and Technology (Guangzhou), China（香港科技大学（广州））；School of Information Technology, Carleton University, Canada（卡尔顿大学信息科技学院）
MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering
Khoury College of Computer Sciences, Northeastern University（东北大学克劳尔计算机科学学院）；New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）
University of Michigan（密歇根大学）；UC Riverside（加州大学河滨分校）；Google Research（谷歌研究院）；University of Southern California（南加州大学）；University of British Columbia（不列颠哥伦比亚大学）
Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models
National Key Laboratory of Radar Signal Processing, Xidian University（西安电子科技大学雷达信号处理国家重点实验室）；McCombs School of Business, The University of Texas at Austin（德克萨斯大学奥斯汀分校麦库姆斯商学院）
