arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.02010cs.CVcs.AI
收藏

用于密集预测任务的多级对比学习

Multi-Level Contrastive Learning for Dense Prediction Task

The University of Hong Kong(香港大学);ByteDance Inc.(字节跳动公司)

Qiushan Guo, Yizhou Yu, Yi Jiang, Jiannan Wu, Zehuan Yuan, Ping Luo

AI总结:

提出多级对比学习(MCL)自监督方法,通过蒙太奇组装多尺度图像构建多级对比损失,学习区域级特征表示,在密集预测任务上显著优于现有方法并减少预训练轮数。

摘要:

在这项工作中,我们提出了用于密集预测任务的多级对比学习(MCL),这是一种用于学习密集预测任务的区域级特征表示的高效自监督方法。我们的方法受到检测中三个关键因素的启发:定位、尺度一致性和识别。为了显式编码绝对位置和尺度信息,我们提出了一种新颖的代理任务,该任务以蒙太奇方式组装多尺度图像以模拟多目标场景。与现有的图像级自监督方法不同,我们的方法构建了一个多级对比损失,将蒙太奇图像的每个子区域视为一个单独实例。我们的方法使神经网络能够学习用于平移和尺度一致性的区域语义表示,同时将预训练轮数减少到与有监督预训练相同。大量实验表明,MCL在各种数据集上始终以显著优势优于最近的最先进方法。特别是,当使用以100个轮数预训练的R50-FPN为主干的Mask R-CNN进行1x计划微调时,MCL在COCO上获得了42.5 AP$^\mathrm{bb}$和38.3 AP$^\mathrm{mk}$。与MoCo相比,我们的方法在性能上超越了4.0 AP$^\mathrm{bb}$和3.1 AP$^\mathrm{mk}$。此外,我们探索了代理任务与下游任务之间的对齐。我们将代理任务扩展到有监督预训练,取得了与自监督学习相似的性能。这一结果证明了代理任务与下游任务之间对齐的重要性,表明我们的方法在自监督设置之外具有更广泛的适用性潜力。

英文摘要:

In this work, we present Multi-Level Contrastive Learning for Dense Prediction Task (MCL), an efficient self-supervised method for learning region-level feature representation for dense prediction tasks. Our method is motivated by the three key factors in detection: localization, scale consistency and recognition. To explicitly encode absolute position and scale information, we propose a novel pretext task that assembles multi-scale images in a montage manner to mimic multi-object scenarios. Unlike the existing image-level self-supervised methods, our method constructs a multi-level contrastive loss that considers each sub-region of the montage image as a singleton. Our method enables the neural network to learn regional semantic representations for translation and scale consistency while reducing pre-training epochs to the same as supervised pre-training. Extensive experiments demonstrate that MCL consistently outperforms the recent state-of-the-art methods on various datasets with significant margins. In particular, MCL obtains 42.5 AP$^\mathrm{bb}$ and 38.3 AP$^\mathrm{mk}$ on COCO with the 1x schedule fintuning, when using Mask R-CNN with R50-FPN backbone pre-trained with 100 epochs. In comparison to MoCo, our method surpasses their performance by 4.0 AP$^\mathrm{bb}$ and 3.1 AP$^\mathrm{mk}$. Furthermore, we explore the alignment between pretext task and downstream tasks. We extend our pretext task to supervised pre-training, which achieves a similar performance to self-supervised learning. This result demonstrates the importance of the alignment between pretext task and downstream tasks, indicating the potential for wider applicability of our method beyond self-supervised settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑