DiffCLIP：差分注意力与CLIP的结合
DiffCLIP: Differential Attention Meets CLIP
AI总结：
本文提出DiffCLIP，将差分注意力机制引入CLIP双编码器，以极少参数和可忽略的计算开销在零样本分类、检索和鲁棒性任务上持续超越基线CLIP模型。
摘要：
我们提出DiffCLIP，一种新颖的视觉-语言模型，将差分注意力机制扩展到CLIP架构中。差分注意力最初是为大型语言模型开发的，用于放大相关上下文并消除噪声信息。在本工作中，我们将该机制集成到CLIP的双编码器（图像和文本）框架中。在仅增加极少参数的情况下，DiffCLIP在图像-文本理解任务上取得了优越的性能。在零样本分类、检索和鲁棒性基准测试中，DiffCLIP始终优于基线CLIP模型。值得注意的是，这些提升带来的计算开销可以忽略不计，表明差分注意力能够在不牺牲效率的前提下显著增强多模态表示。代码可在 https://github.com/hammoudhasan/DiffCLIP 获取。
英文摘要：
We propose DiffCLIP, a novel vision-language model that extends the differential attention mechanism to CLIP architectures. Differential attention was originally developed for large language models to amplify relevant context while canceling out noisy information. In this work, we integrate this mechanism into CLIP's dual encoder (image and text) framework. With minimal additional parameters, DiffCLIP achieves superior performance on image-text understanding tasks. Across zero-shot classification, retrieval, and robustness benchmarks, DiffCLIP consistently outperforms baseline CLIP models. Notably, these gains come with negligible computational overhead, demonstrating that differential attention can significantly enhance multi-modal representations without sacrificing efficiency. Code can be found at https://github.com/hammoudhasan/DiffCLIP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
