arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.04796cs.CV
收藏

Local-Prompt:面向少样本分布外检测的可扩展局部提示

Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Centre for Artificial Intelligence and Robotics, HKISI-CAS(香港中文大学(深圳)人工智能与机器人中心(HKISI-CAS));Southern University of Science and Technology(南方科技大学)

Fanhu Zeng, Zhen Cheng, Fei Zhu, Hongxin Wei, Xu-Yao Zhang

AI总结:

针对少样本OOD检测中局部信息利用不足的问题,提出Local-Prompt方法,冻结全局提示并引入局部提示的粗到细调优范式,通过负增强和区域正则化提升性能,在ImageNet-1k上4-shot调优平均FPR95降低5.17%。

摘要:

分布外(OOD)检测旨在区分已知类别之外的异常样本,在实际场景中已受到广泛关注。近年来,视觉-语言模型(VLM)的出现提升了通过少样本调优来增强VLM的OOD检测的兴趣。然而,现有方法主要侧重于优化全局提示,忽略了针对异常样本的局部信息的精细利用。受此启发,我们冻结全局提示,并引入Local-Prompt,一种新颖的从粗到细的调优范式,通过局部提示强调区域增强。我们的方法包含两个核心组件:全局提示引导的负增强和局部提示增强的区域正则化。前者利用冻结的粗粒度全局提示作为引导线索来引入负增强,从而利用局部异常知识。后者采用可训练的局部提示和区域正则化来有效捕获局部信息,助力异常识别。我们还提出了与区域相关的度量,以增强OOD检测的丰富性。此外,由于我们的方法仅探索增强局部提示,因此在推理过程中可以无缝地与训练好的全局提示集成,以提升性能。全面的实验证明了我们方法的有效性和潜力。值得注意的是,在具有挑战性的ImageNet-1k数据集上进行4-shot调优时,我们的方法相比最先进的方法将平均FPR95降低了5.17%,甚至超过了先前方法的16-shot结果。代码已发布在https://github.com/AuroraZengfh/Local-Prompt。

英文摘要:

Out-of-Distribution (OOD) detection, aiming to distinguish outliers from known categories, has gained prominence in practical scenarios. Recently, the advent of vision-language models (VLM) has heightened interest in enhancing OOD detection for VLM through few-shot tuning. However, existing methods mainly focus on optimizing global prompts, ignoring refined utilization of local information with regard to outliers. Motivated by this, we freeze global prompts and introduce Local-Prompt, a novel coarse-to-fine tuning paradigm to emphasize regional enhancement with local prompts. Our method comprises two integral components: global prompt guided negative augmentation and local prompt enhanced regional regularization. The former utilizes frozen, coarse global prompts as guiding cues to incorporate negative augmentation, thereby leveraging local outlier knowledge. The latter employs trainable local prompts and a regional regularization to capture local information effectively, aiding in outlier identification. We also propose regional-related metric to empower the enrichment of OOD detection. Moreover, since our approach explores enhancing local prompts only, it can be seamlessly integrated with trained global prompts during inference to boost the performance. Comprehensive experiments demonstrate the effectiveness and potential of our method. Notably, our method reduces average FPR95 by 5.17% against state-of-the-art method in 4-shot tuning on challenging ImageNet-1k dataset, even outperforming 16-shot results of previous methods. Code is released at https://github.com/AuroraZengfh/Local-Prompt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑