arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.02735cs.LG
收藏

OOD-Chameleon:用于OOD泛化的算法选择是可学习的吗?

OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?

EPFL(瑞士联邦理工学院);Idiap Research Institute(IDIAP研究所)

Liangze Jiang, Damien Teney

AI总结:

本文提出OOD-Chameleon,将OOD泛化的训练算法选择建模为多标签分类,通过数据集特征在无需训练模型的情况下预测高性能算法,为算法选择和适用性提供了新见解。

摘要:

分布外(OOD)泛化具有挑战性,因为分布偏移以多种形式出现。存在大量算法来解决特定场景,但无需试错地为合适的数据集选择正确的训练算法是困难的。实际上,现实世界的应用通常涉及多种类型的偏移及其组合,这在理论上很难分析。方法。本研究探索了为OOD泛化学习训练算法选择的可能性。我们提出了一个概念验证(OOD-Chameleon),将算法选择表述为对候选算法的多标签分类,并在包含多种偏移的数据集集合上进行训练。我们评估了OOD-Chameleon基于数据集特征对未见过的偏移和数据集进行算法排序的能力,即与传统模型选择不同,它不需要先训练模型。发现。大量实验表明,学习到的选择器能在合成、视觉和语言任务中识别出高性能算法。进一步检查表明,它学习了非平凡的决策规则,为现有算法的适用性提供了新见解。总体而言,这种新方法为更好地利用和理解现有大量OOD泛化算法开辟了可能。

英文摘要:

Out-of-distribution (OOD) generalization is challenging because distribution shifts come in many forms. Numerous algorithms exist to address specific settings, but choosing the right training algorithm for the right dataset without trial and error is difficult. Indeed, real-world applications often involve multiple types and combinations of shifts that are hard to analyze theoretically. Method. This work explores the possibility of learning the selection of a training algorithm for OOD generalization. We propose a proof of concept (OOD-Chameleon) that formulates the selection as a multi-label classification over candidate algorithms, trained on a dataset of datasets representing a variety of shifts. We evaluate the ability of OOD-Chameleon to rank algorithms on unseen shifts and datasets based only on dataset characteristics, i.e., without training models first, unlike traditional model selection. Findings. Extensive experiments show that the learned selector identifies high-performing algorithms across synthetic, vision, and language tasks. Further inspection shows that it learns non-trivial decision rules, which provide new insights into the applicability of existing algorithms. Overall, this new approach opens the possibility of better exploiting and understanding the plethora of existing algorithms for OOD generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑