CLIP-Decoder：基于多模态CLIP对齐表示的零样本多标签分类
CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation
AI总结：
该研究提出基于ML-Decoder的CLIP-Decoder方法，通过多模态表示学习与图像-文本嵌入对齐及组合损失优化，在零样本和广义零样本多标签分类任务上分别取得3.9%和近2.3%的性能提升。
摘要：
多标签分类是一项在众多实际应用中广泛使用的核心任务。多标签零样本学习是一种可将图像分类到多个无训练数据的未见类别的方法，而在通用零样本场景中，测试集可能包含已观测类别。CLIP-Decoder是基于当前最优的ML-Decoder（基于注意力的分类头）提出的全新方法。我们在CLIP-Decoder中引入多模态表示学习，利用文本编码器提取文本特征，图像编码器提取图像特征。此外，我们通过将图像嵌入和词嵌入对齐至相同维度，并使用由分类损失与CLIP损失组成的组合损失比较二者的表示，来最小化语义不匹配问题。该策略性能优于其他方法，我们使用CLIP-Decoder在零样本多标签分类任务上取得了前沿结果。在零样本学习多标签分类任务中，我们的方法相比现有方法性能绝对提升3.9%。此外，在广义零样本学习多标签分类任务中，我们的方法实现了近2.3%的显著提升。
英文摘要：
Multi-label classification is an essential task utilized in a wide variety of real-world applications. Multi-label zero-shot learning is a method for classifying images into multiple unseen categories for which no training data is available, while in general zero-shot situations, the test set may include observed classes. The CLIP-Decoder is a novel method based on the state-of-the-art ML-Decoder attention-based head. We introduce multi-modal representation learning in CLIP-Decoder, utilizing the text encoder to extract text features and the image encoder for image feature extraction. Furthermore, we minimize semantic mismatch by aligning image and word embeddings in the same dimension and comparing their respective representations using a combined loss, which comprises classification loss and CLIP loss. This strategy outperforms other methods and we achieve cutting-edge results on zero-shot multilabel classification tasks using CLIP-Decoder. Our method achieves an absolute increase of 3.9% in performance compared to existing methods for zero-shot learning multi-label classification tasks. Additionally, in the generalized zero-shot learning multi-label classification task, our method shows an impressive increase of almost 2.3%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
