大规模预训练是良好域泛化的秘诀吗？
Is Large-Scale Pretraining the Secret to Good Domain Generalization?
AI总结：
本文提出对齐假设，认为域泛化性能取决于预训练中图像与文本嵌入的对齐程度，并通过划分预训练内外数据验证了现有方法的局限，强调需超越预训练对齐的泛化方法。
摘要：
多源域泛化（DG）是在多个源域上训练并在未见过的目标域上实现高分类性能的任务。近期方法将来自网络规模预训练骨干网络的稳健特征与从源数据学习到的新特征相结合，这显著提升了基准测试结果。然而，目前尚不清楚DG微调方法是否随时间变得更好，还是基准性能的提升仅仅是更强预训练的产物。先前研究表明，与预训练数据的感知相似性与零样本性能相关，但我们发现这种效应在DG设置中有限。相反，我们假设预训练中具有感知相似的数据并不足够；而是这些数据被学习得有多好决定了性能。这引导我们提出对齐假设，该假设指出最终的DG性能将高，当且仅当图像和类别标签文本嵌入的对齐度高时。我们的实验证实了对齐假设是正确的，并将其用作分析工具，通过将评估数据分为预训练内（IP）和预训练外（OOP）来评估在DomainBed数据集上评估的现有DG方法。我们表明，所有评估的DG方法在DomainBed-OOP上表现挣扎，而近期方法在DomainBed-IP上表现出色。综合起来，我们的发现强调了需要能够超越预训练对齐进行泛化的DG方法。
英文摘要：
Multi-Source Domain Generalization (DG) is the task of training on multiple source domains and achieving high classification performance on unseen target domains. Recent methods combine robust features from web-scale pretrained backbones with new features learned from source data, and this has dramatically improved benchmark results. However, it remains unclear if DG finetuning methods are becoming better over time, or if improved benchmark performance is simply an artifact of stronger pre-training. Prior studies have shown that perceptual similarity to pre-training data correlates with zero-shot performance, but we find the effect limited in the DG setting. Instead, we posit that having perceptually similar data in pretraining is not enough; and that it is how well these data were learned that determines performance. This leads us to introduce the Alignment Hypothesis, which states that the final DG performance will be high if and only if alignment of image and class label text embeddings is high. Our experiments confirm the Alignment Hypothesis is true, and we use it as an analysis tool of existing DG methods evaluated on DomainBed datasets by splitting evaluation data into In-pretraining (IP) and Out-of-pretraining (OOP). We show that all evaluated DG methods struggle on DomainBed-OOP, while recent methods excel on DomainBed-IP. Put together, our findings highlight the need for DG methods which can generalize beyond pretraining alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
