arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21035cs.CVcs.MM
收藏

不只是存在与否:无需微调即可使CLIP理解否定性视觉描述

Not Just What's There: Enabling CLIP to Comprehend Negated Visual Descriptions Without Fine-tuning

Junhao Xiao, Zhiyu Wu, Hao Lin, Yi Chen, Yahui Liu, Xiaoran Zhao, Zixu Wang, Zejiang He

AI总结:

CLIPGlasses通过双阶段设计提升CLIP对否定性视觉描述的理解能力,无需微调,增强跨领域泛化性能。

摘要:

视觉-语言模型(VLMs)如CLIP在理解否定方面存在困难,通常将肯定和否定表达嵌入相似(例如,将

英文摘要:

Vision-Language Models (VLMs) like CLIP struggle to understand negation, often embedding affirmatives and negatives similarly (e.g., matching "no dog" with dog images). Existing methods refine negation understanding via fine-tuning CLIP's text encoder, risking overfitting. In this work, we propose CLIPGlasses, a plug-and-play framework that enhances CLIP's ability to comprehend negated visual descriptions. CLIPGlasses adopts a dual-stage design: a Lens module disentangles negated semantics from text embeddings, and a Frame module predicts context-aware repulsion strength, which is integrated into a modified similarity computation to penalize alignment with negated semantics, thereby reducing false positive matches. Experiments show that CLIP equipped with CLIPGlasses achieves competitive in-domain performance and outperforms state-of-the-art methods in cross-domain generalization. Its superiority is especially evident under low-resource conditions, indicating stronger robustness across domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑