Okapi:通过统计匹配更好地概括
Okapi: Generalising Better by Making Statistical Matches Match
摘要:
我们提出了Okapi,一个简单,高效,通用的方法,强大的半监督学习基于在线统计匹配。我们的方法使用基于最近邻的匹配过程来生成一致性损失的跨域视图,同时消除统计离群值。为了在运行时和内存有效的方式进行在线匹配,我们借鉴了自我监督的文献,并结合一个存储体与一个缓慢移动的动量编码器。一致性损失被应用在特征空间内,而不是在预测分布上,使得该方法对所讨论的模态和任务都是不可知的。我们在WILDS 2.0数据集上进行了实验,Sagawa等人,这大大扩展了模式,应用程序的范围,以及可用于研究和基准测试现实世界的无监督适应的转变。与Sagawa等人相反,我们表明,它实际上是可以利用额外的未标记的数据,以改善经验风险最小化(ERM)的结果与正确的方法。我们的方法在iWildCam(多类分类任务)和IdentityMap(回归任务)图像数据集以及CivilComments(二进制分类任务)文本数据集的分布外(OOD)概括方面优于基线方法。此外,从定性的角度来看,我们表明从学习的编码器获得的匹配是强语义相关的。我们论文的代码可在https://github.com/wearepal/okapi/上公开获取。
英文摘要:
We propose Okapi, a simple, efficient, and general method for robust semi-supervised learning based on online statistical matching. Our method uses a nearest-neighbours-based matching procedure to generate cross-domain views for a consistency loss, while eliminating statistical outliers. In order to perform the online matching in a runtime- and memory-efficient way, we draw upon the self-supervised literature and combine a memory bank with a slow-moving momentum encoder. The consistency loss is applied within the feature space, rather than on the predictive distribution, making the method agnostic to both the modality and the task in question. We experiment on the WILDS 2.0 datasets Sagawa et al., which significantly expands the range of modalities, applications, and shifts available for studying and benchmarking real-world unsupervised adaptation. Contrary to Sagawa et al., we show that it is in fact possible to leverage additional unlabelled data to improve upon empirical risk minimisation (ERM) results with the right method. Our method outperforms the baseline methods in terms of out-of-distribution (OOD) generalisation on the iWildCam (a multi-class classification task) and PovertyMap (a regression task) image datasets as well as the CivilComments (a binary classification task) text dataset. Furthermore, from a qualitative perspective, we show the matches obtained from the learned encoder are strongly semantically related. Code for our paper is publicly available at https://github.com/wearepal/okapi/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。