arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.07785cs.LG
收藏

HYPO:超球面分布外泛化

HYPO: Hyperspherical Out-of-Distribution Generalization

University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Amazon(亚马逊公司)

Haoyue Bai, Yifei Ming, Julian Katz-Samuels, Yixuan Li

AI总结:

本文提出HYPO框架,在超球面空间通过类内对齐与类间分离原则学习领域不变表示,提升分布外泛化性能,理论证明其改进泛化界限,实验优于基线。

摘要:

分布外(OOD)泛化对于部署在现实世界中的机器学习模型至关重要。然而,实现这一目标可能具有根本性的挑战,因为它需要具备在不同领域或环境中学习不变特征的能力。在本文中,我们提出了一种新颖的框架HYPO(超球面OOD泛化),该框架可证明地在超球面空间中学习领域不变表示。具体而言,我们的超球面学习算法以类内变异和类间分离原则为指导——确保来自同一类(跨不同训练领域)的特征与其类原型紧密对齐,而不同类原型则被最大化分离。我们进一步提供了关于我们的原型学习目标如何改进OOD泛化界限的理论论证。通过在具有挑战性的OOD基准上进行的大量实验,我们证明了我们的方法优于竞争基线并实现了卓越的性能。代码可在https://github.com/deeplearning-wisc/hypo获取。

英文摘要:

Out-of-distribution (OOD) generalization is critical for machine learning models deployed in the real world. However, achieving this can be fundamentally challenging, as it requires the ability to learn invariant features across different domains or environments. In this paper, we propose a novel framework HYPO (HYPerspherical OOD generalization) that provably learns domain-invariant representations in a hyperspherical space. In particular, our hyperspherical learning algorithm is guided by intra-class variation and inter-class separation principles -- ensuring that features from the same class (across different training domains) are closely aligned with their class prototypes, while different class prototypes are maximally separated. We further provide theoretical justifications on how our prototypical learning objective improves the OOD generalization bound. Through extensive experiments on challenging OOD benchmarks, we demonstrate that our approach outperforms competitive baselines and achieves superior performance. Code is available at https://github.com/deeplearning-wisc/hypo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑