Know-Thy-Not：双编码器视觉-语言模型中无需训练的定向否定处理
Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models
AI总结：
针对双编码器视觉-语言模型无法处理否定的问题，提出无需训练的KTN方法，引入图像侧拒绝维度，在9种骨干模型上使COCO-Confusion的mAP获显著提升。
摘要：
双编码器视觉-语言模型在否定处理上存在系统性缺陷，对“没有狗的猫”和“有狗的猫”的评分几乎完全相同。我们证明该缺陷是单嵌入框架的固有问题：在对比训练下，单嵌入方法最多只能忽略被否定的概念（稀释），却永远无法惩罚其存在（拒绝）。我们提出Know-Thy-Not（KTN），一种无需训练的方法，通过引入图像侧拒绝维度直接在视觉表征中惩罚被否定的概念；还推出首个需要真正拒绝才能成功的基准COCO-Confusion。在9种骨干模型上，KTN使COCO-Confusion的mAP较普通视觉-语言模型提升最高达25.6，较现有最强方法提升9.6。
英文摘要：
Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
