Existing prompt learning methods have shown certain capabilities in Out-of-Distribution (OOD) detection, but the lack of OOD images in the target dataset in their training can lead to mismatches between OOD images and In-Distribution (ID) categories, resulting in a high false positive rate. To address this issue, we introduce a novel OOD detection method, named 'NegPrompt', to learn a set of negative prompts, each representing a negative connotation of a given class label, for delineating the boundaries between ID and OOD images. It learns such negative prompts with ID data only, without any reliance on external outlier data. Further, current methods assume the availability of samples of all ID classes, rendering them ineffective in open-vocabulary learning scenarios where the inference stage can contain novel ID classes not present during training. In contrast, our learned negative prompts are transferable to novel class labels. Experiments on various ImageNet benchmarks show that NegPrompt surpasses state-of-the-art prompt-learning-based OOD detection methods and maintains a consistent lead in hard OOD detection in closed- and open-vocabulary classification scenarios. Code is available at https://github.com/mala-lab/negprompt.
Towards Highly Transferable Vision-Language Attack via Semantic-Augmented Dynamic Contrastive Interaction
School of Artificial Intelligence and Computer Science, Jiangnan University(江南大学人工智能与计算机科学学院);Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey(Surrey 大学视觉、语音和信号处理中心)
Prompt-driven Transferable Adversarial Attack on Person Re-Identification with Attribute-aware Textual Inversion
School of Artificial Intelligence and Robotics, Hunan University(湖南大学人工智能与机器人学院);National Engineering Research Center of Robot Visual Perception and Control Technology(机器人视觉感知与控制技术国家工程研究中心);College of Information Science and Engineering, Hunan Normal University(湖南师范大学信息科学与工程学院)
One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所人工智能安全国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学)
VerifyPrompt: Towards Text-to-Image Models Verification Behind Black-Box APIs
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China(电子科技大学智能协同计算实验室,电子科技大学);School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院,电子科技大学)
Transferable Decoding with Visual Entities for Zero-Shot Image Captioning
Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Tencent(腾讯);Shanghai Jiao Tong University(上海交通大学);Southern University of Science and Technology(南方科技大学);The University of Hong Kong(香港大学)
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model
Northwestern Polytechnical University(西北工业大学);Research & Development Institute of Northwestern Polytechnical University in Shenzhen(西北工业大学深圳研究院);Xidian University(西安电子科技大学)
Guiding What Not to Generate: Automated Negative Prompting for Text-Image Alignment
Department of Electrical and Computer Engineering, Seoul National University(电子与计算机工程系,首尔国立大学);Amazon(亚马逊);IPAI, AIIS, ASRI, INMC, and ISRC, Seoul National University(IPAI、AIIS、ASRI、INMC 和 ISRC,首尔国立大学)
Orthogonal Negative Guidance in Attention Feature Space for Text-to-Image Generation
Interdisciplinary Program in Artificial Intelligence, Seoul National University(人工智能交叉学科项目,首尔国立大学);Research Institute for Convergence Science, Seoul National University(融合科学研究所,首尔国立大学);Artificial Intelligence Institute, Seoul National University(人工智能研究所,首尔国立大学);Department of Intelligence and Information, Seoul National University(智能与信息系,首尔国立大学);Daegu Gyeongbuk Institute of Science and Technology(大邱庆北科学技术院);Samsung Advanced Institute of Technology, Samsung Electronics Co., Ltd(三星先进技术研究所,三星电子公司)
Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
Mehta Family School of Data Science and Artificial Intelligence(梅hta家族数据科学与人工智能学院);Indian Institute of Technology, Roorkee(印度理工学院罗奥克学院);Department of Civil Engineering(土木工程系);Department of Electronics and Communication(电子与通信系)
Towards Transferable Defense Against Malicious Image Edits
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (CAS)(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of China Academy of Sciences(中国科学院大学);School of Computer Science, China University of Geosciences(中国地质大学(武汉)计算机学院)
Bayesian Prompt Learning for Image-Language Model Generalization
University of Amsterdam(阿姆斯特丹大学);Samsung AI Cambridge(三星人工智能剑桥研究院);Technical University of Iasi(雅西技术大学);University of Trento(特伦托大学);Queen Mary University of London(伦敦玛丽女王大学)
Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidance
FPT Software AI Center(FPT软件AI中心);TU Wien(维也纳技术大学);AIT GmbH(AIT有限公司);Imperial College London(伦敦帝国学院);Physical Intelligence(物理智能公司);University of Arkansas(阿肯色大学);Ton Duc Thang University(同塔大学);University of Liverpool(利物浦大学)
ClickSAM: Fine-tuning Segment Anything Model using click prompts for ultrasound image segmentation
The Hockaday School(霍克黛学校);Liberty High School(自由高中);The University of Texas at Dallas(德克萨斯大学达拉斯分校);The University of Texas Southwestern Medical Center at Dallas(德克萨斯大学达拉斯西南医学中心)