近年来，大规模视觉与语言模型的进展在零样本学习任务中取得了显著进步。CoOp和CoCoOp等方法表明，用可学习向量（即提示学习）替代手工设计的提示可以提升性能。然而，这些模型往往难以泛化到完全未见过的类别。传统零样本学习技术受益于多种数据增强策略，但提示学习主要聚焦于基于文本的修改，基于图像的增强潜力在很大程度上尚未被探索。在本工作中，我们探索了图像级增强（尤其是引入属性特定变化的增强）如何支持和增强提示学习。我们的分析考察了这些增强与软提示框架之间的交互，揭示了它们在改善泛化方面的潜力。我们还识别出现有方法（如CoCoOp）的一个局限性：它们没有为学习聚焦于语义上有意义的视觉特征的提示提供明确指导。为解决这一问题，我们提出了Adding Attributes to Prompt Learning（AAPL），这是一种新方法，通过引入对抗性token嵌入，将增强引入的表面视觉变化与类别相关的语义表示解耦。这种解耦使学习到的提示能够集中于与目标类别对齐的视觉判别性特征。我们在十一个基准数据集上进行了全面实验，AAPL在少样本、零样本、跨数据集和域泛化设置中一致优于现有方法。我们的源代码公开于：https://github.com/Gahyeonkim09/AAPL
英文摘要：
Recent advances in large-scale vision and language models have led to significant progress in zero-shot learning tasks. Methods such as CoOp and CoCoOp have shown that replacing handcrafted prompts with learnable vectors, known as prompt learning, can result in improved performance. However, these models often struggle to generalize to entirely unseen categories. While traditional zero-shot learning techniques benefit from various data augmentation strategies, prompt learning has primarily focused on text-based modifications, leaving the potential of image-based augmentation largely unexplored. In this work, we explore how image-level augmentations, particularly those that introduce attribute-specific variations, can support and enhance prompt learning. Our analysis examines the interaction between these augmentations and soft prompt frameworks, revealing their potential to improve generalization. We also identify a limitation in existing methods, such as CoCoOp, which do not provide explicit guidance for learning prompts that focus on semantically meaningful visual features. To address this, we propose Adding Attributes to Prompt Learning, AAPL, a novel method that introduces adversarial token embeddings to decouple superficial visual variations introduced by augmentation from class-relevant semantic representations. This decoupling enables the learned prompts to concentrate on visually discriminative features that align with the target categories. We conduct comprehensive experiments on eleven benchmark datasets, and AAPL consistently outperforms existing methods across few-shot, zero-shot, cross-dataset, and domain generalization settings. Our source code is publicly available at: https://github.com/Gahyeonkim09/AAPL
Gotta: Generative Few-shot Question Answering by Prompt-based Cloze Data Augmentation
University of California, Los Angeles（加州大学洛杉矶分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Univ of Technology Sydney（悉尼科技大学）；Amazon Search（亚马逊搜索）
Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Harbin Institute of Technology（哈尔滨工业大学）
MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
UCAS-Terminus AI Lab, University of Chinese Academic of Sciences（中国科学院大学Terminus AI实验室，中国科学院大学）；AIM Lab, University of Amsterdam（阿姆斯特丹大学AIM实验室）；Jarvis Research Center, Tencent Youtu Lab（腾讯优图实验室 Jarvis 研究中心）；Central Research Institue, United Imaging Healthcare Co., Ltd（联合影像医疗科技有限公司中央研究所）
Seeing is Believing: Aligning Prompt Rewriting with Visual Anchors for Text-to-Image Generation
Peking University（北京大学）；Tencent（腾讯）；Dalian University of Technology（大连理工大学）；Nanyang Technological University（南洋理工大学）；University of Cambridge（剑桥大学）；Zhejiang University（浙江大学）
SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentation
INESC-ID（葡萄牙国家系统与计算机工程研究所）；Instituto Superior Técnico（高等技术学院）；University of Lisbon（里斯本大学）；University of Copenhagen（哥本哈根大学）；Pioneer Center for AI（先锋人工智能中心）
Performance Improvement of Language-Queried Audio Source Separation Based on Caption Augmentation From Large Language Models for DCASE Challenge 2024 Task 9
Gwangju Institute of Science and Technology（光州科学技术院）；Aunion AI, Co. Ltd.（Aunion AI有限公司）
SenseCF: LLM-Prompted Counterfactuals for Intervention and Sensor Data Augmentation
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；School of Nutritional Sciences and Wellness, University of Arizona（亚利桑那大学营养科学与健康学院）
USE: A Unified Self-Ensembling Framework for Test-Time Prompt Tuning
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所国家智能机器人实验室及模式识别实验室）；Nanjing University（南京大学）
When Color-Space Decoupling Meets Diffusion for Adverse-Weather Image Restoration
School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）；School of Electrical Engineering, Southeast University（电气工程学院，东南大学）；School of Intelligence Science and Technology, Nanjing University（智能科学与技术学院，南京大学）
Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
School of Computer Science and Engineering（计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications（新一代人工智能技术及其交叉应用重点实验室）；Southeast University（东南大学）；School of Intelligence Science and Engineering（智能科学与工程学院）
Speak & Spell: LLM-Driven Controllable Phonetic Error Augmentation for Robust Dialogue State Tracking
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea（人工智能研究生院，POSTECH，韩国）；Department of Computer Science and Engineering, POSTECH, Republic of Korea（计算机科学与工程系，POSTECH，韩国）
