文本即图像：为多标签分类学习可迁移的 Adapter
Text as Image: Learning Transferable Adapter for Multi-Label Classification
AI总结：
针对提示微调在开放词汇多标签分类中候选标签过多导致训练昂贵的问题，本文提出可在文本模态训练、迁移到视觉模态识别标签的 adapter，并用随机扰动与 LLM 指令生成构建无人工数据的自动流程。
摘要：
预训练视觉-语言模型显著加速了开放世界概念识别的进展。近期，其令人瞩目的零样本能力已通过提示微调迁移到多标签图像分类，从而能够以开放词汇表的方式发现新标签。然而，这一范式存在不可忽视的训练成本，并且当候选标签数量庞大时，其计算开销会高到令人难以承受。为解决这一问题，我们注意到视觉-语言预训练将图像和文本对齐到一个统一的嵌入空间中，这使得 adapter 网络有可能在视觉模态中识别标签，同时在文本模态中接受训练。为增强这种跨模态迁移能力，我们提出了一种简单而有效的方法，称为随机扰动；该方法在训练过程中通过噪声扰动文本嵌入，使 adapter 能够搜索潜在的视觉嵌入，从而在视觉模态中取得更好的性能。此外，我们还引入了一种有效方法，利用大语言模型进行遵循多标签指令的文本生成。通过这种方式，我们构建了一条完全自动化的视觉标签识别流程，而无需依赖任何人工数据。在公开基准上进行的大量实验表明，我们的方法在多种多标签分类任务中具有优越性。
英文摘要：
Pre-trained vision-language models have notably accelerated progress of open-world concept recognition. Their impressive zero-shot ability has recently been transferred to multi-label image classification via prompt tuning, enabling to discover novel labels in an open-vocabulary manner. However, this paradigm suffers from non-trivial training costs, and becomes computationally prohibitive for a large number of candidate labels. To address this issue, we note that vision-language pre-training aligns images and texts in a unified embedding space, making it potential for an adapter network to identify labels in visual modality while be trained in text modality. To enhance such cross-modal transfer ability, a simple yet effective method termed random perturbation is proposed, which enables the adapter to search for potential visual embeddings by perturbing text embeddings with noise during training, resulting in better performance in visual modality. Furthermore, we introduce an effective approach to employ large language models for multi-label instruction-following text generation. In this way, a fully automated pipeline for visual label recognition is developed without relying on any manual data. Extensive experiments on public benchmarks show the superiority of our method in various multi-label classification tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
