视觉语言模型中标签噪声提示调优的内在梯度抑制
Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models
AI总结：
提出双Softmax提示调优（DSPT），通过内在梯度抑制机制自适应压制高错误噪声样本的梯度，实现标签噪声下的鲁棒提示调优。
摘要：
对比视觉语言模型如CLIP展现出显著的零样本泛化能力。然而，提示调优对标签噪声高度敏感，因为错误标记的样本会产生不成比例的大梯度，可能压倒预训练先验。我们认为，由于CLIP已经提供了接近最优的初始化，适应过程应本质上是保守的，特别是在噪声设置中常见的极端梯度更新情况下。为此，我们提出了双Softmax提示调优（DSPT），一种无需超参数的内在梯度抑制方法。通过应用顺序概率归一化，DSPT诱导出一个自适应饱和区，该区域抑制来自高错误噪声样本的梯度，同时保持信息性更新。我们还提供了关于该机制如何实现自适应抑制的理论分析和实证证据。这种设计将传统上作为训练瓶颈的“梯度消失”转化为标签噪声提示调优的原则性噪声过滤盾牌。大量实验证实，这种简单、即插即用的设计在各种噪声基准上实现了最先进的鲁棒性，优于具有复杂架构和手工调整超参数的方法。
英文摘要：
Contrastive vision-language models like CLIP exhibit remarkable zero-shot generalization. However, prompt tuning remains highly sensitive to label noise, as mislabeled samples generate disproportionately large gradients that can overwhelm pre-trained priors. We argue that because CLIP already provides a near-optimal initialization, adaptation should be inherently conservative, particularly against the extreme gradient updates common in noisy settings. To this end, we propose Double-Softmax Prompt Tuning (DSPT), a hyperparameter-free method for intrinsic gradient suppression. By applying a sequential probabilistic normalization, DSPT induces a self-adaptive saturation zone that suppresses gradients from high-error noisy samples while maintaining informative updates. We also provide both theoretical analysis and empirical evidence about how this mechanism achieves adaptive suppression. This design transforms ``gradient vanishing'', traditionally a training bottleneck, into a principled noise-filtering shield for label-noise prompt tuning. Extensive experiments confirm that this simple, drop-in design achieves state-of-the-art robustness across various noisy benchmarks, outperforming methods with complex architectures and handcrafted hyperparameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
