野外领域泛化：将分类与领域感知表示解耦
Domain Generalization in-the-Wild: Disentangling Classification from Domain-Aware Representations
AI总结：
针对CLIP在野外领域泛化中性能下降的问题，提出CLIP-DCA方法，通过增强领域感知并解耦分类与领域特征，在33个数据集上显著提升OOD性能。
摘要：
评估CLIP等基础模型的领域泛化（DG）具有挑战性，因为网络规模的预训练数据可能已覆盖许多现有基准。因此，当前的DG评估可能既不够具有挑战性，也未充分测试真正未见的数据场景。为了更好地评估CLIP在野外DG（即CLIP遇到具有挑战性的未见数据的场景）中的性能，我们考虑了两种方法：（1）在ImageNet上微调CLIP后，在33个具有量化分布外（OOD）分数的多样化数据集上进行评估；（2）使用遗忘（unlearning）使CLIP“忘记”某些领域作为近似。我们观察到，CLIP在更多OOD数据集上的性能显著下降。为解决此问题，我们提出了CLIP-DCA（将分类与增强的领域感知表示解耦）。我们的方法基于以下观察：虽然标准的领域不变性损失旨在使表示具有领域不变性，但这可能对基础模型有害，因为它强制丢弃对泛化有益的领域感知表示。相反，我们假设增强领域感知是基础模型中有效领域不变分类的前提。CLIP-DCA使用单独的领域头（domain head）和合成生成的多样化领域数据来识别并增强CLIP编码器内的领域感知。同时，它通过与领域特征解耦来鼓励领域不变分类。在此具有挑战性的评估中，CLIP-DCA相比现有方法显示出显著改进，尤其是在更OOD的数据集上。
英文摘要：
Evaluating domain generalization (DG) for foundational models like CLIP is challenging, as web-scale pretraining data potentially covers many existing benchmarks. Consequently, current DG evaluation may neither be sufficiently challenging nor adequately test genuinely unseen data scenarios. To better assess the performance of CLIP on DG in-the-wild, a scenario where CLIP encounters challenging unseen data, we consider two approaches: (1) evaluating on 33 diverse datasets with quantified out-of-distribution (OOD) scores after fine-tuning CLIP on ImageNet, and (2) using unlearning to make CLIP `forget' some domains as an approximation. We observe that CLIP's performance deteriorates significantly on more OOD datasets. To address this, we present CLIP-DCA (Disentangling Classification from enhanced domain Aware representations). Our approach is motivated by the observation that while standard domain invariance losses aim to make representations domain-invariant, this can be harmful to foundation models by forcing the discarding of domain-aware representations beneficial for generalization. We instead hypothesize that enhancing domain awareness is a prerequisite for effective domain-invariant classification in foundation models. CLIP-DCA identifies and enhances domain awareness within CLIP's encoders using a separate domain head and synthetically generated diverse domain data. Simultaneously, it encourages domain-invariant classification through disentanglement from the domain features. CLIP-DCA shows significant improvements within this challenging evaluation compared to existing methods, particularly on datasets that are more OOD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
