回归骨干：重新发现骨干在域泛化中的作用
Back-to-Bones: Rediscovering the Role of Backbones in Domain Generalization
AI总结：
本文系统分析不同骨干网络对域泛化能力的影响，发现骨干性能与DG能力线性相关，采用强骨干加数据增强的朴素ERM即可超越现有DG算法，提出以骨干为中心的新研究范式。
摘要：
域泛化（DG）研究深度学习模型泛化到训练分布之外的能力。在过去十年中，文献中充斥着声称能获得更抽象、更鲁棒的数据表示以应对域偏移的训练方法。最近的研究为DG提供了一个可复现的基准，指出了朴素经验风险最小化（ERM）相对于现有算法的有效性。然而，研究者们仍在使用相同的过时特征提取器，且尚未关注不同骨干网络的影响。在本文中，我们回归骨干，对其内在泛化能力进行了全面分析，这一能力迄今被研究界忽视。我们评估了多种特征提取器，从标准残差解决方案到基于Transformer的架构，发现大规模单域分类准确率与DG能力之间存在明显的线性相关性。我们的大量实验表明，采用有竞争力的骨干网络并结合有效的数据增强，朴素ERM优于最近的DG解决方案，并达到了最先进的准确率。此外，我们的额外定性研究揭示，新型骨干网络对同类样本给出更相似的表征，在特征空间中将不同域分开。这种泛化能力的提升为DG算法留下了很小的空间。它提出了一种研究该问题的新范式，将骨干网络置于聚光灯下，并鼓励在其之上开发一致的算法。代码可在 https://github.com/PIC4SeR/Back-to-Bones 获取。
英文摘要：
Domain Generalization (DG) studies the capability of a deep learning model to generalize to out-of-training distributions. In the last decade, literature has been massively filled with training methodologies that claim to obtain more abstract and robust data representations to tackle domain shifts. Recent research has provided a reproducible benchmark for DG, pointing out the effectiveness of naive empirical risk minimization (ERM) over existing algorithms. Nevertheless, researchers persist in using the same outdated feature extractors, and no attention has been given to the effects of different backbones yet. In this paper, we start back to the backbones proposing a comprehensive analysis of their intrinsic generalization capabilities, which so far have been ignored by the research community. We evaluate a wide variety of feature extractors, from standard residual solutions to transformer-based architectures, finding an evident linear correlation between large-scale single-domain classification accuracy and DG capability. Our extensive experimentation shows that by adopting competitive backbones in conjunction with effective data augmentation, plain ERM outperforms recent DG solutions and achieves state-of-the-art accuracy. Moreover, our additional qualitative studies reveal that novel backbones give more similar representations to same-class samples, separating different domains in the feature space. This boost in generalization capabilities leaves marginal room for DG algorithms. It suggests a new paradigm for investigating the problem, placing backbones in the spotlight and encouraging the development of consistent algorithms on top of them. The code is available at https://github.com/PIC4SeR/Back-to-Bones.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
