Review of Large Vision Models and Visual Prompt Engineering
Northwestern Polytechnical University(西北工业大学);The University of Georgia(佐治亚大学);City University of Hong Kong(香港城市大学);Shaanxi Normal University(陕西师范大学);University of Electronic Science and Technology of China(电子科技大学);The University of Texas at Arlington(德克萨斯大学阿灵顿分校);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院)
Jiaqi Wang, Zhengliang Liu, Lin Zhao, Zihao Wu, Chong Ma, Sigang Yu, Haixing Dai, Qiushi Yang, Yiheng Liu, Songyao Zhang, Enze Shi, Yi Pan, Tuo Zhang, Dajiang Z… 展开作者
Jiaqi Wang, Zhengliang Liu, Lin Zhao, Zihao Wu, Chong Ma, Sigang Yu, Haixing Dai, Qiushi Yang, Yiheng Liu, Songyao Zhang, Enze Shi, Yi Pan, Tuo Zhang, Dajiang Zhu, Xiang Li, Xi Jiang, Bao Ge, Yixuan Yuan, Dinggang Shen, Tianming Liu, Shu Zhang
Visual prompt engineering is a fundamental technology in the field of visual and image Artificial General Intelligence, serving as a key component for achieving zero-shot capabilities. As the development of large vision models progresses, the importance of prompt engineering becomes increasingly evident. Designing suitable prompts for specific visual tasks has emerged as a meaningful research direction. This review aims to summarize the methods employed in the computer vision domain for large vision models and visual prompt engineering, exploring the latest advancements in visual prompt engineering. We present influential large models in the visual domain and a range of prompt engineering methods employed on these models. It is our hope that this review provides a comprehensive and systematic description of prompt engineering methods based on large visual models, offering valuable insights for future researchers in their exploration of this field.
Prompt Tuning with Soft Context Sharing for Vision-Language Models
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA(中国科学院自动化研究所多模态人工智能系统国家重点实验室);Engineering Laboratory for Intelligent Industrial Vision, CASIA(中国科学院自动化研究所智能工业视觉工程实验室);Research Center of Aerospace Information, CASIA(中国科学院自动化研究所航天信息研究中心)
A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models
University of Oxford(牛津大学);Ludwig Maximilian University of Munich(慕尼黑大学);Google Research(谷歌研究院);Munich Center for Machine Learning(慕尼黑机器学习中心);University of California, Santa Barbara(加州大学圣巴巴拉分校)
Pro-tuning: Unified Prompt Tuning for Vision Tasks
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Huawei Cloud & AI(华为云与计算BG);CAS Centre for Artificial Intelligence and Robotics, HK Institute of Science and Innovation(中国科学院香港创新研究院人工智能与机器人创新中心)
Learning Visual Prompts for Guiding the Attention of Vision Transformers
School of Computation, Information and Technology(计算、信息与技术学院);Technical University of Munich(慕尼黑工业大学);AI Innovation, Data:Lab Munich(慕尼黑人工智能创新数据实验室);Volkswagen AG(大众汽车集团);Department of Engineering Science(工程科学系);University of Oxford(牛津大学)
A Framework for Critical Evaluation of Text-to-Image Models: Integrating Art Historical Analysis, Artistic Exploration, and Critical Prompt Engineering
School of Fine Arts, University of Ioannina(约阿尼纳大学美术学院)
Prompt Estimation from Prototypes for Federated Prompt Tuning of Vision Transformers
Department of Computational and Data Sciences, Indian Institute of Science(计算与数据科学系,印度科学研究院);Accenture, Japan(日本Accenture公司);Google, India(印度Google公司);Indian Institute of Science(印度科学研究院)
Transitive Vision-Language Prompt Learning for Domain Generalization
Xiamen University(厦门大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳))
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models
Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Tsinghua University(清华大学);New York University(纽约大学)
MoPD: Mixture-of-Prompts Distillation for Vision-Language Models
Department of Computer Science and Engineering, Southern University of Science and Technology(计算机科学与工程系,南方科技大学);School of Mathematical and Computer Sciences, University of Adelaide(数学与计算机科学学院,阿德莱德大学)