School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Academy for Advanced Interdisciplinary Studies, Peking University（北京大学前沿交叉学科研究院）；College of Future Technology, Peking University（北京大学未来技术学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络科学与工程学院）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
Wenqiang Zu, Shenghao Xie, Qing Zhao, Guoqi Li, Lei Ma
Foundation models pre-trained on large-scale data have been widely witnessed to achieve success in various natural imaging downstream tasks. Parameter-efficient fine-tuning (PEFT) methods aim to adapt foundation models to new domains by updating only a small portion of parameters in order to reduce computational overhead. However, the effectiveness of these PEFT methods, especially in cross-domain few-shot scenarios, e.g., medical image analysis, has not been fully explored. In this work, we facilitate the study of the performance of PEFT when adapting foundation models to medical image classification tasks. Furthermore, to alleviate the limitations of prompt introducing ways and approximation capabilities on Transformer architectures of mainstream prompt tuning methods, we propose the Embedded Prompt Tuning (EPT) method by embedding prompt tokens into the expanded channels. We also find that there are anomalies in the feature space distribution of foundation models during pre-training process, and prompt tuning can help mitigate this negative impact. To explain this phenomenon, we also introduce a novel perspective to understand prompt tuning: Prompt tuning is a distribution calibrator. And we support it by analyzing patch-wise scaling and feature separation operations contained in EPT. Our experiments show that EPT outperforms several state-of-the-art fine-tuning methods by a significant margin on few-shot medical image classification tasks, and completes the fine-tuning process within highly competitive time, indicating EPT is an effective PEFT method. The source code is available at github.com/zuwenqiang/EPT.
SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentation
The Chinese University of Hong Kong（香港中文大学）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Hupan Lab（湖畔实验室）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；Southern University of Science and Technology（南方科技大学）
University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Tulane University（路易斯安那州立大学）；Carnegie Mellon University（卡内基梅隆大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Georgia Institute of Technology（佐治亚理工学院）
E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning
Rochester Institute of Technology（罗切斯特理工学院）；Meta AI；University of Florida（佛罗里达大学）；Purdue University（普渡大学）；Zhejiang University（浙江大学）；BIGAI（北京通用人工智能研究院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering
Khoury College of Computer Sciences, Northeastern University（东北大学克劳尔计算机科学学院）；New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）
Federated Prompt-Tuning with Heterogeneous and Incomplete Multimodal Client Data
Institute of AI Innovation and Societal Impact, Hanoi University of Science and Technology（人工智能创新与社会影响研究院，河内科学技术大学）；EPFL（瑞士联邦理工学院）；Griffith University（格里菲斯大学）；Washington State University（华盛顿州立大学）
Pro-tuning: Unified Prompt Tuning for Vision Tasks
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Huawei Cloud & AI（华为云与计算BG）；CAS Centre for Artificial Intelligence and Robotics, HK Institute of Science and Innovation（中国科学院香港创新研究院人工智能与机器人创新中心）
Historical Test-time Prompt Tuning for Vision Foundation Models
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；University of Chinese Academy of Sciences（中国科学院大学）
Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained Model
Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院）；Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院医学系）；NUS Graduate School, National University of Singapore（新加坡国立大学研究生院）
Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection
Zhejiang University（浙江大学）；The Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）；Singapore Management University（新加坡管理大学）
Prompt Estimation from Prototypes for Federated Prompt Tuning of Vision Transformers
Department of Computational and Data Sciences, Indian Institute of Science（计算与数据科学系，印度科学研究院）；Accenture, Japan（日本Accenture公司）；Google, India（印度Google公司）；Indian Institute of Science（印度科学研究院）
Exploring Interpretability for Visual Prompt Tuning with Cross-layer Concepts
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Microsoft Research Asia（微软亚洲研究院）；School of Telecommunication and Engineering, Xidian University（西安电子科技大学电信与工程学院）
MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
UCAS-Terminus AI Lab, University of Chinese Academic of Sciences（中国科学院大学Terminus AI实验室，中国科学院大学）；AIM Lab, University of Amsterdam（阿姆斯特丹大学AIM实验室）；Jarvis Research Center, Tencent Youtu Lab（腾讯优图实验室 Jarvis 研究中心）；Central Research Institue, United Imaging Healthcare Co., Ltd（联合影像医疗科技有限公司中央研究所）
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model
Northwestern Polytechnical University（西北工业大学）；Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Xidian University（西安电子科技大学）
Guizhou University, Guiyang, China（贵州大学）；Harbin Institute of Technology, China（哈尔滨工业大学）；The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳））
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））；University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校（UIUC））；Imperial College London（伦敦帝国学院）
GA2-CLIP: Generic Attribute Anchor for Efficient Prompt Tuningin Video-Language Models
Shandong Provincial Key Laboratory of New Power Distribution & Utilization Technology and Equipment, School of Electrical and Electronic Engineering, Shandong University of Technology（山东省新型电力分布与利用技术及设备重点实验室，山东理工大学电气与电子工程学院）；School of Computer Science and Technology, Shandong University of Technology（山东理工大学计算机科学与技术学院）；School of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；Pillar of Information Systems Technology and Design, Singapore University of Technology and Design（新加坡科技设计大学信息系统技术与设计 pillar）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）
FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models
University of Trento（特伦托大学）；University of Bergamo（贝拉姆奥大学）；Indian Institute of Technology Bombay（印度班加罗尔理工学院）；LNMIIT Jaipur（斋普尔LNMIIT）；The University of Queensland（昆士兰大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）
University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Tulane University（杜兰大学）；Brown University（布朗大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Carnegie Mellon University（卡内基梅隆大学）
Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models
National Key Laboratory of Radar Signal Processing, Xidian University（西安电子科技大学雷达信号处理国家重点实验室）；McCombs School of Business, The University of Texas at Austin（德克萨斯大学奥斯汀分校麦库姆斯商学院）
