arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.12448cs.LGcs.AIcs.CVstat.ML
收藏

生成数据总是有助于对比学习吗?

Do Generated Data Always Help Contrastive Learning?

Peking University(北京大学);MIT(麻省理工学院);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);School of Mathematical Sciences, Peking University(北京大学数学科学学院)

Yifei Wang, Jizhe Zhang, Yisen Wang

AI总结:

本文发现生成数据可能损害对比学习,揭示数据膨胀与增强的互补关系,提出无需额外成本的AdaInf策略,在CIFAR-10上创下94.70%的线性准确率新纪录。

摘要:

对比学习(Contrastive Learning, CL)已成为无监督视觉表示学习中最成功的范式之一,然而它通常依赖于密集的手动数据增强。随着生成模型(尤其是扩散模型)的兴起,生成接近真实数据分布的逼真图像的能力已得到广泛认可。这些生成的高质量图像已被成功应用于增强对比表示学习,这一技术被称为“数据膨胀”(data inflation)。然而,我们发现生成的数据(即使来自像DDPM这样的优秀扩散模型)有时甚至可能损害对比学习。我们从数据膨胀和数据增强两个角度研究了这一失败背后的原因。我们首次揭示了互补的作用:更强的数据膨胀应伴随较弱的数据增强,反之亦然。我们还通过推导数据膨胀下的泛化界,为这些现象提供了严格的理论解释。基于这些见解,我们提出了自适应膨胀(Adaptive Inflation, AdaInf),一种纯粹以数据为中心的策略,不引入任何额外计算成本。在基准数据集上,AdaInf能为各种对比学习方法带来显著改进。值得注意的是,在不使用外部数据的情况下,AdaInf在CIFAR-10上使用SimCLR获得了94.70%的线性准确率,创下了超越许多复杂方法的新纪录。代码可在https://github.com/PKU-ML/adainf获取。

英文摘要:

Contrastive Learning (CL) has emerged as one of the most successful paradigms for unsupervised visual representation learning, yet it often depends on intensive manual data augmentations. With the rise of generative models, especially diffusion models, the ability to generate realistic images close to the real data distribution has been well recognized. These generated high-equality images have been successfully applied to enhance contrastive representation learning, a technique termed ``data inflation''. However, we find that the generated data (even from a good diffusion model like DDPM) may sometimes even harm contrastive learning. We investigate the causes behind this failure from the perspective of both data inflation and data augmentation. For the first time, we reveal the complementary roles that stronger data inflation should be accompanied by weaker augmentations, and vice versa. We also provide rigorous theoretical explanations for these phenomena via deriving its generalization bounds under data inflation. Drawing from these insights, we propose Adaptive Inflation (AdaInf), a purely data-centric strategy without introducing any extra computation cost. On benchmark datasets, AdaInf can bring significant improvements for various contrastive learning methods. Notably, without using external data, AdaInf obtains 94.70% linear accuracy on CIFAR-10 with SimCLR, setting a new record that surpasses many sophisticated methods. Code is available at https://github.com/PKU-ML/adainf.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑