Out-of-distribution (OOD) generalization is critical for machine learning models deployed in the real world. However, achieving this can be fundamentally challenging, as it requires the ability to learn invariant features across different domains or environments. In this paper, we propose a novel framework HYPO (HYPerspherical OOD generalization) that provably learns domain-invariant representations in a hyperspherical space. In particular, our hyperspherical learning algorithm is guided by intra-class variation and inter-class separation principles -- ensuring that features from the same class (across different training domains) are closely aligned with their class prototypes, while different class prototypes are maximally separated. We further provide theoretical justifications on how our prototypical learning objective improves the OOD generalization bound. Through extensive experiments on challenging OOD benchmarks, we demonstrate that our approach outperforms competitive baselines and achieves superior performance. Code is available at https://github.com/deeplearning-wisc/hypo.
Raising the Bar in Graph OOD Generalization: Invariant Learning Beyond Explicit Environment Modeling
The School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);The School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Domain-Specific Risk Minimization for Out-of-Distribution Generalization
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation(自动化研究所);Microsoft Research Asia(微软亚洲研究院);The University of Sydney(悉尼大学)
When is invariance useful in an Out-of-Distribution Generalization problem ?
Preferred Networks
分布外(OOD)泛化问题的目标是训练一个能在所有环境下泛化的预测器。该领域的流行方法使用这样的假设,即这种预测器应该是一个\textit{invariant predictor},它捕获在不同环境中保持恒定的机制。虽然这些方法在各种案例研究中取得了实验成功,但这一假设的理论验证仍有很大的空间。本文提出了一组新的理论条件所需的不变预测,以实现OOD最优性。我们的理论不仅适用于非线性情形,而且推广了\citet{rojas2018invariant}中的必要条件。我们还从我们的理论中推导出帧间梯度对齐算法,并在MNIST衍生的基准数据集上以及在\citet{aubinlinear}提出的三个\textit{Invariance Unit Tests}中的两个上证明了其竞争力。
Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures
Univ Rennes(雷恩大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);IRISA(雷恩信息与自动化研究所);Université Jean Monnet Saint-Etienne(圣艾蒂安让·莫奈大学);Institut d Optique Graduate School(光学研究所研究生院);Laboratoire Hubert Curien(于贝尔·居里实验室);Institut Universitaire de France (IUF)(法国大学研究院);ServiceNow Research(ServiceNow研究院)