arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.08243cs.CVcs.AIcs.LGeess.IV
收藏

基于联合嵌入预测架构的图像自监督学习

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture

Meta AI (FAIR)(Meta人工智能(基础人工智能研究院));McGill University(麦吉尔大学);Mila, Quebec AI Institute(米拉-魁北克人工智能研究所);New York University(纽约大学)

Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, Nicolas Ballas

AI总结:

本文提出 I-JEPA,一种基于联合嵌入预测的非生成式自监督图像学习方法,通过语义级目标块掩码与信息丰富的上下文块设计,结合 ViT 实现高效可扩展训练,在多种下游任务中取得优异表现。

摘要:

本文展示了一种不依赖手工设计数据增强来学习高度语义化图像表示的方法。我们提出了基于图像的联合嵌入预测架构(I-JEPA),这是一种用于图像自监督学习的非生成式方法。I-JEPA 的核心思想很简单:从单个上下文块出发,预测同一图像中多个目标块的表示。引导 I-JEPA 产生语义表示的一个核心设计选择是掩码策略;具体而言,关键在于(a)采样具有足够大尺度(语义性)的目标块,以及(b)使用信息量足够丰富(空间分布充分)的上下文块。实验表明,当与 Vision Transformers 结合使用时,我们发现 I-JEPA 具有高度的可扩展性。例如,我们使用 16 块 A100 GPU 在不到 72 小时内在 ImageNet 上训练了一个 ViT-Huge/14,在从线性分类到物体计数和深度预测的广泛任务范围内取得了强大的下游性能。

英文摘要:

This paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations. We introduce the Image-based Joint-Embedding Predictive Architecture (I-JEPA), a non-generative approach for self-supervised learning from images. The idea behind I-JEPA is simple: from a single context block, predict the representations of various target blocks in the same image. A core design choice to guide I-JEPA towards producing semantic representations is the masking strategy; specifically, it is crucial to (a) sample target blocks with sufficiently large scale (semantic), and to (b) use a sufficiently informative (spatially distributed) context block. Empirically, when combined with Vision Transformers, we find I-JEPA to be highly scalable. For instance, we train a ViT-Huge/14 on ImageNet using 16 A100 GPUs in under 72 hours to achieve strong downstream performance across a wide range of tasks, from linear classification to object counting and depth prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑