Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Centre for Artificial Intelligence and Robotics, HKISI-CAS(香港中文大学(深圳)人工智能与机器人中心(HKISI-CAS));Southern University of Science and Technology(南方科技大学)
Out-of-Distribution (OOD) detection, aiming to distinguish outliers from known categories, has gained prominence in practical scenarios. Recently, the advent of vision-language models (VLM) has heightened interest in enhancing OOD detection for VLM through few-shot tuning. However, existing methods mainly focus on optimizing global prompts, ignoring refined utilization of local information with regard to outliers. Motivated by this, we freeze global prompts and introduce Local-Prompt, a novel coarse-to-fine tuning paradigm to emphasize regional enhancement with local prompts. Our method comprises two integral components: global prompt guided negative augmentation and local prompt enhanced regional regularization. The former utilizes frozen, coarse global prompts as guiding cues to incorporate negative augmentation, thereby leveraging local outlier knowledge. The latter employs trainable local prompts and a regional regularization to capture local information effectively, aiding in outlier identification. We also propose regional-related metric to empower the enrichment of OOD detection. Moreover, since our approach explores enhancing local prompts only, it can be seamlessly integrated with trained global prompts during inference to boost the performance. Comprehensive experiments demonstrate the effectiveness and potential of our method. Notably, our method reduces average FPR95 by 5.17% against state-of-the-art method in 4-shot tuning on challenging ImageNet-1k dataset, even outperforming 16-shot results of previous methods. Code is released at https://github.com/AuroraZengfh/Local-Prompt.
PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection
East China Normal University(华东师范大学);Chongqing Institute of East China Normal University(华东师范大学重庆研究院);The Navy Military Medical University(海军军医大学);Xiamen University(厦门大学)
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Harbin Engineering University(哈尔滨工程大学);Key Laboratory of Advanced Marine Communication and Information Technology(先进海洋通信与信息技术重点实验室);University of Toronto(多伦多大学);Kanagawa University(神奈川大学)
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts
Inria(法国国家信息与自动化研究所);Univ Rennes(雷恩大学);CNRS(法国国家科学研究中心);IRISA(法国雷恩信息与自动化系统研究所);Institute of Advanced Research in Artificial Intelligence (IARAI)(高级人工智能研究所(IARAI))
DiffuSAM: Diffusion-Based Prompt-Free SAM2 for Few-Shot and Source-Free Medical Image Segmentation
School of Electrical Engineering, Tel Aviv University, Israel(特拉维夫大学电气工程学院);School of Biomedical Engineering, Tel Aviv University, Israel(特拉维夫大学生物医学工程学院)
University of Science and Technology of China(中国科学技术大学);Center for Research on Intelligent Perception and Computing, NLPR, CASIA(中国科学院自动化研究所模式识别国家重点实验室智能感知与计算研究中心);Nanyang Technological University(南洋理工大学);University of Chinese Academy of Sciences(中国科学院大学)
Taiyuan University of Technology(太原理工大学);University of Science and Technology of China(中国科学技术大学);Second Hospital of Shanxi Medical University(山西医科大学第二医院)
Prompting Decision Transformer for Few-Shot Policy Generalization
Carnegie Mellon University(卡内基梅隆大学);University of Montreal(蒙特利尔大学);Mila(米拉);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);Massachusetts Institute of Technology(麻省理工学院);UMass Amherst(马萨诸塞大学阿默斯特分校)
Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs
ICG, TU Graz(格拉茨工业大学ICG研究所);CDL-EML(CDL-EML中心);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);JKU(约翰内斯开普勒大学);IBM Research(IBM研究院);Weizmann Institute of Science(魏茨曼科学研究所);University of Bonn(波恩大学)