BREEDS：亚群转移的基准
BREEDS: Benchmarks for Subpopulation Shift
摘要：
我们开发了一种方法来评估模型对亚群迁移的鲁棒性-具体来说，就是它们推广到训练过程中未观察到的新数据亚群的能力。我们的方法利用现有数据集的类结构来控制组成训练和测试分布的数据子群体。这使我们能够合成现实的分布偏移，其来源可以精确控制和特点，在现有的大规模数据集。将这种方法应用于ImageNet数据集，我们创建了一套不同粒度的子群体迁移基准。然后，我们通过获得人类基线来验证相应的变化是易于处理的。最后，我们利用这些基准来衡量标准模型架构的敏感性以及现成的训练时间鲁棒性干预的有效性。代码和数据可在https://github.com/MadryLab/BREEDS-Benchmarks获得。
英文摘要：
We develop a methodology for assessing the robustness of models to subpopulation shift---specifically, their ability to generalize to novel data subpopulations that were not observed during training. Our approach leverages the class structure underlying existing datasets to control the data subpopulations that comprise the training and test distributions. This enables us to synthesize realistic distribution shifts whose sources can be precisely controlled and characterized, within existing large-scale datasets. Applying this methodology to the ImageNet dataset, we create a suite of subpopulation shift benchmarks of varying granularity. We then validate that the corresponding shifts are tractable by obtaining human baselines for them. Finally, we utilize these benchmarks to measure the sensitivity of standard model architectures as well as the effectiveness of off-the-shelf train-time robustness interventions. Code and data available at https://github.com/MadryLab/BREEDS-Benchmarks .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
