arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2003.04298cs.CV
收藏

对比自监督学习中的变换式作文

On Compositions of Transformations in Contrastive Self-Supervised Learning

Facebook AI Research(Facebook人工智能研究院);University of Oxford(牛津大学)

Mandela Patrick, Yuki M. Asano, Polina Kuznetsova, Ruth Fong, João F. Henriques, Geoffrey Zweig, Andrea Vedaldi

摘要:

在图像域中,可以通过噪声对比学习将不变性引入到内容保持变换来学习优秀的表示。在本文中,我们将对比学习推广到一组更广泛的变换,以及它们的组合,寻求不变性或独特性。我们表明,这是不是立即明显的现有方法,如Simplified可以扩展到这样做。相反,我们引入了一些正式的要求,所有的对比配方必须满足,并提出了一个实际的建设,满足这些要求。为了最大限度地扩大这种分析的范围,我们将噪声对比公式的所有组成部分表示为数据(GDT)的某些广义变换的选择,包括数据采样。然后,我们将视频视为数据的一个例子,其中可以应用各种各样的转换,考虑到额外的模态-我们分析音频和文本-以及时间维度。我们发现,对某些变换保持不变,对其他变换保持独特,对于学习有效的视频表示,大幅提高多个基准的最新技术水平,甚至超越监督预训练至关重要。

英文摘要:

In the image domain, excellent representations can be learned by inducing invariance to content-preserving transformations via noise contrastive learning. In this paper, we generalize contrastive learning to a wider set of transformations, and their compositions, for which either invariance or distinctiveness is sought. We show that it is not immediately obvious how existing methods such as SimCLR can be extended to do so. Instead, we introduce a number of formal requirements that all contrastive formulations must satisfy, and propose a practical construction which satisfies these requirements. In order to maximise the reach of this analysis, we express all components of noise contrastive formulations as the choice of certain generalized transformations of the data (GDTs), including data sampling. We then consider videos as an example of data in which a large variety of transformations are applicable, accounting for the extra modalities -- for which we analyze audio and text -- and the dimension of time. We find that being invariant to certain transformations and distinctive to others is critical to learning effective video representations, improving the state-of-the-art for multiple benchmarks by a large margin, and even surpassing supervised pretraining.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑