利用锚定属性推进文本提示学习
Advancing Textual Prompt Learning with Anchored Attributes
AI总结：
本文提出 ATPrompt，通过将属性标记融入软提示，把文本提示从类别中心形式扩展为属性-类别混合形式，并利用可微属性搜索增强图像与未知类别的对齐，以极低计算成本提升视觉-语言模型在下游任务中的表现。
摘要：
基于文本的提示学习方法主要采用多个可学习的软提示和硬类别标记以级联方式作为文本输入，旨在对齐图像与文本（类别）空间以完成下游任务。然而，当前的训练仅限于将图像与预定义的已知类别对齐，无法与未知类别建立关联。在本工作中，我们提出利用通用属性作为桥梁，以增强图像与未知类别之间的对齐。具体而言，我们为视觉-语言模型引入了一种属性锚定的文本提示学习方法，命名为 ATPrompt。该方法通过将多个属性标记纳入可学习的软提示中，将软提示的学习空间从原始的一维类别层级扩展到多维属性层级。通过这一修改，我们将文本提示从以类别为中心的形式转变为属性-类别混合形式。此外，我们引入了一种简单直接的可微属性搜索方法，以识别适合下游任务的代表性属性。作为一种易于使用的即插即用技术，ATPrompt 可以无缝替换基于文本方法中现有的基础提示格式，以可忽略的计算成本提供普遍的性能提升。在 11 个数据集上的大量实验验证了我们方法的有效性。代码已公开于 https://github.com/zhengli97/ATPrompt。
英文摘要：
Textual-based prompt learning methods primarily employ multiple learnable soft prompts and hard class tokens in a cascading manner as text inputs, aiming to align image and text (category) spaces for downstream tasks. However, current training is restricted to aligning images with predefined known categories and cannot be associated with unknown categories. In this work, we propose utilizing universal attributes as a bridge to enhance the alignment between images and unknown categories. Specifically, we introduce an Attribute-anchored Textual Prompt learning method for vision-language models, named ATPrompt. This approach expands the learning space of soft prompts from the original one-dimensional category level into the multi-dimensional attribute level by incorporating multiple attribute tokens into the learnable soft prompts. Through this modification, we transform the text prompt from a category-centric form to an attribute-category hybrid form. Additionally, we introduce a straightforward differentiable attribute search method to identify representative and suitable attributes for downstream tasks. As an easy-to-use plug-in technique, ATPrompt can seamlessly replace the existing basic prompt format in textual-based methods, providing general improvements at a negligible computational cost. Extensive experiments across 11 datasets validate the effectiveness of our method. Code is publicly available at https://github.com/zhengli97/ATPrompt.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
