arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.00566cs.CVcs.AIcs.LG
收藏

随机位置嵌入改进掩码图像建模

Stochastic positional embeddings improve masked image modeling

Tel Aviv University(特拉维夫大学);UC Berkeley(加州大学伯克利分校);Meta AI (FAIR)(元人工智能研究院(FAIR));New York University(纽约大学)

Amir Bar, Florian Bordes, Assaf Shocher, Mahmoud Assran, Pascal Vincent, Nicolas Ballas, Trevor Darrell, Amir Globerson, Yann LeCun

AI总结:

本文提出随机位置嵌入StoP,通过从高斯分布采样掩码令牌位置来引入位置不确定性,减少对位置特征的过拟合,从而提升掩码图像建模在下游任务中的性能。

摘要:

掩码图像建模(MIM)是一种有前景的自监督学习方法,能够从未标注图像中学习。尽管近来取得了成功,但通过MIM学习良好的表示仍然具有挑战性,因为它要求在准确的位置预测正确的语义内容。例如,给定一张不完整的狗的图像,我们可以猜测那里有一条尾巴,但无法确定其确切位置。在这项工作中,我们提出通过使用随机位置嵌入(StoP)将位置不确定性纳入MIM。具体来说,我们以从高斯分布中采样的随机掩码令牌位置为条件来调节模型。StoP减少了对位置特征的过拟合,并引导模型学习对位置不确定性更鲁棒的特征。定量上,StoP在多种下游任务上提升了下游MIM性能,包括使用ViT-B在ImageNet线性探测上提升+1.7%,以及使用1%数据时ViT-H提升+2.5%。

英文摘要:

Masked Image Modeling (MIM) is a promising self-supervised learning approach that enables learning from unlabeled images. Despite its recent success, learning good representations through MIM remains challenging because it requires predicting the right semantic content in accurate locations. For example, given an incomplete picture of a dog, we can guess that there is a tail, but we cannot determine its exact location. In this work, we propose to incorporate location uncertainty into MIM by using stochastic positional embeddings (StoP). Specifically, we condition the model on stochastic masked token positions drawn from a Gaussian distribution. StoP reduces overfitting to location features and guides the model toward learning features that are more robust to location uncertainties. Quantitatively, StoP improves downstream MIM performance on a variety of downstream tasks, including $+1.7\%$ on ImageNet linear probing using ViT-B, and $+2.5\%$ for ViT-H using $1\%$ of the data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑