DivIL：揭示并解决分布外泛化中的过度不变性问题
DivIL: Unveiling and Addressing Over-Invariance for Out-of- Distribution Generalization
AI总结：
针对分布外泛化中强不变约束导致的过度不变性问题，提出DivIL方法，结合无监督对比学习和随机掩蔽机制，在12个数据集和6个模型上验证了有效性。
摘要：
分布外泛化是一个常见问题，期望模型在不同分布下，甚至远离训练数据时也能表现良好。解决此问题的一种流行方法是不变性学习（IL），通过在训练期间添加强约束，使模型专注于不变特征而非虚假特征。然而，强不变约束存在一些潜在陷阱。由于多样环境的数量有限以及特征空间中的过度正则化，在缓解虚假相关性的同时，可能导致不变特征中重要细节的丢失，即过度不变性，这也会降低泛化性能。我们从理论上定义了过度不变性，并观察到该问题出现在各种经典IL方法中。为了缓解此问题，我们提出了一种简单的方法——多样化不变性学习（DivIL），通过添加无监督对比学习和随机掩蔽机制来补偿不变约束，该方法可应用于各种IL方法。此外，我们在12个数据集和6个经典模型上跨多种模态进行了实验，验证了我们的过度不变性见解以及DivIL框架的有效性。我们的代码可在https://github.com/kokolerk/DivIL获取。
英文摘要：
Out-of-distribution generalization is a common problem that expects the model to perform well in the different distributions even far from the train data. A popular approach to addressing this issue is invariant learning (IL), in which the model is compiled to focus on invariant features instead of spurious features by adding strong constraints during training. However, there are some potential pitfalls of strong invariant constraints. Due to the limited number of diverse environments and over-regularization in the feature space, it may lead to a loss of important details in the invariant features while alleviating the spurious correlations, namely the over-invariance, which can also degrade the generalization performance. We theoretically define the over-invariance and observe that this issue occurs in various classic IL methods. To alleviate this issue, we propose a simple approach Diverse Invariant Learning (DivIL) by adding the unsupervised contrastive learning and the random masking mechanism compensatory for the invariant constraints, which can be applied to various IL methods. Furthermore, we conduct experiments across multiple modalities across 12 datasets and 6 classic models, verifying our over-invariance insight and the effectiveness of our DivIL framework. Our code is available at https://github.com/kokolerk/DivIL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
