自我汤合：无标签的模型汤烹饪
Self-Soupervision: Cooking Model Soups without Labels
AI总结：
提出Self-Soupervision方法，将模型汤技术扩展到自监督学习，通过使用无标签数据混合不同自监督算法训练的参数，提升模型鲁棒性和准确性。
摘要：
模型汤是参数的一种奇特且异常有效的组合。它们将一个模型（底汤）微调成多个模型（配料），然后将它们的参数混合回一个模型（汤）以改进预测。虽然所有已知的汤都需要监督学习，并在标记数据上优化相同的损失，但我们的Self-Soupervision配方将汤推广到自监督学习（SSL）。我们的Self-Souping允许我们在新的数据源上调味配料，例如来自迁移任务的无标签数据或来自鲁棒性迁移的数据。我们表明，在损坏的测试数据上进行Self-Souping，然后回到未损坏的训练数据上进行微调，可以将鲁棒性提升+3.5%（ImageNet-C）和+7%（LAION-C）。Self-Soupervision还解锁了无数SSL算法，以烹饪更鲁棒汤所需的各种配料。我们首次表明，配料可以在其SSL超参数上有所不同——更令人惊讶的是，在其SSL算法上也可以不同。我们烹饪了MAE、MoCoV3、MMCR和LeJEPA配料的汤，这些汤比任何单个SSL配料都更准确。
英文摘要：
Model soups are strange and strangely effective combinations of parameters. They take a model (the stock), fine-tune it into multiple models (the ingredients), and then mix their parameters back into one model (the soup) to improve predictions. While all known soups require supervised learning, and optimize the same loss on labeled data, our recipes for Self-Soupervision generalize soups to self-supervised learning (SSL). Our Self-Souping lets us flavor ingredients on new data sources, e.g. from unlabeled data from a task for transfer or from a shift for robustness. We show that Self-Souping on corrupted test data, then fine-tuning back on uncorrupted train data, boosts robustness by +3.5% (ImageNet-C) and +7% (LAION-C). Self-Soupervision also unlocks countless SSL algorithms to cook the diverse ingredients needed for more robust soups. We show for the first time that ingredients can differ in their SSL hyperparameters -- and more surprisingly, in their SSL algorithms. We cook soups of MAE, MoCoV3, MMCR, and LeJEPA ingredients that are more accurate than any single SSL ingredient.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
