National Engineering Research Center for Software Engineering, Peking University(北京大学软件工程国家工程研究中心);Huawei Cloud & AI(华为云计算与人工智能);School of Mathematical Sciences, Peking University(北京大学数学科学学院);National Key Lab of General AI, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室);Peng Cheng Laboratory(鹏城实验室)
Despite recent competitive performance across a range of vision tasks, vision Transformers still have an issue of heavy computational costs. Recently, vision prompt learning has provided an economic solution to this problem without fine-tuning the whole large-scale models. However, the efficiency of existing models are still far from satisfactory due to insertion of extensive prompts blocks and trick prompt designs. In this paper, we propose an efficient vision model named impLicit vIsion prOmpt tuNing (LION), which is motivated by deep implicit models with stable memory costs for various complex tasks. In particular, we merely insect two equilibrium implicit layers in two ends of the pre-trained main backbone with parameters in the backbone frozen. Moreover, we prune the parameters in these two layers according to lottery hypothesis. The performance obtained by our LION are promising on a wide range of datasets. In particular, our LION reduces up to 11.5% of training parameter numbers while obtaining higher performance compared with the state-of-the-art baseline VPT, especially under challenging scenes. Furthermore, we find that our proposed LION had a good generalization performance, making it an easy way to boost transfer learning in the future.
EPIPTrack: Rethinking Prompt Modeling with Explicit and Implicit Prompts for Multi-Object Tracking
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University(吉林大学教育部长春符号计算与知识工程重点实验室);Yangtze University(扬子大学)
AOEPT: Breaking the Implicit Modality-Reduction Bottleneck in Modality-Missing Prompt Tuning
University of Electronic Science and Technology of China(电子科技大学);Intelligent Digital Media Technology Key Laboratory of Sichuan Province(四川省智能数字媒体技术重点实验室)
Position: Towards Implicit Prompt For Text-To-Image Models
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Osaka University(大阪大学);The University of Hong Kong(香港大学);Research Institute of Tsinghua University in Shenzhen(清华大学深圳研究院)
Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models
Columbia University(哥伦比亚大学);University of Oxford(牛津大学);LMU Munich(慕尼黑大学);Siemens AG(西门子股份公司);Konrad Zuse School of Excellence in Reliable AI (relAI)(康拉德·楚泽卓越可靠人工智能学院(relAI))
Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);The University of Hong Kong(香港大学);SECE, Shenzhen Graduate School, Peking University(北京大学深圳研究生院电子与计算机工程学院);Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);Research Institute, Sun Yat-sen University, Shenzhen(中山大学深圳研究院)
Implicit and Explicit Language Guidance for Diffusion-based Visual Perception
School of Electrical and Information Engineering, Tianjin University(天津大学电气与信息工程学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);School of Big Data and Software Engineering, Chongqing University(重庆大学大数据与软件学院)
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models
MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Sony(索尼);Weizmann Institute of Science(魏茨曼科学研究院);JKU(约翰纳斯堡大学);Tübingen AI Center(图宾根人工智能中心);UVA(乌得勒支大学);UNSW(新南威尔士大学);TU Graz(格拉茨技术大学);MIT-IBM(麻省理工学院-IBM)
UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation
Institute of Artificial Intelligence, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室人工智能研究院);College of Artificial Intelligence, Tsinghua University(清华大学人工智能学院);Security Department, Alibaba Group(阿里巴巴集团安全部);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-Sen University(中山大学深圳校区网络科学与技术学院)
Prompt Sensitivity in Vision-Language Grounding: How Small Changes in Wording Affect Object Detection
Department of Aerospace Engineering, Indian Institute of Technology Bombay(印度理工学院孟买分校航空航天工程系);Department of Electrical Engineering, Indian Institute of Technology Bombay(印度理工学院孟买分校电气工程系)
Few-Shot Adversarial Prompt Learning on Vision-Language Models
School of Automation(自动化学院);Beijing Institute of Technology(北京理工大学);Sydney AI Centre(悉尼人工智能中心);University of Sydney(悉尼大学);Department of Computer Science(计算机科学系);Hong Kong Baptist University(香港浸会大学)