arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.13674cs.CVcs.AI
收藏

扩散课程:通过图像引导扩散实现从合成到真实的数据课程

Diffusion Curriculum: Synthetic-to-Real Data Curriculum via Image-Guided Diffusion

University of Maryland, College Park(马里兰大学学院公园分校)

Yijun Liang, Shweta Bhardwaj, Tianyi Zhou

AI总结:

针对低质量或稀缺数据训练难题,提出图像引导扩散的DisCL课程,通过调整引导水平生成合成数据谱并聚焦困难样本,在iWildCam和ImageNet-LT上显著提升分类准确率。

摘要:

低质量或稀缺的数据在实践中给深度神经网络的训练带来了重大挑战。虽然经典的数据增强无法贡献非常不同的新数据,但扩散模型通过文本引导提示生成高质量和多样化的合成数据,为构建自我进化的AI打开了一扇新的大门。然而,仅文本引导无法控制合成图像与原始图像的接近程度,导致分布外数据对模型性能有害。为克服这一限制,我们研究图像引导以实现合成图像与真实图像之间的插值谱。在更强的图像引导下,生成的图像与训练数据相似但难以学习;而在较弱的图像引导下,合成图像对模型更容易,但与原始数据的分布差距更大。生成的完整数据谱使我们能够构建一种新颖的“扩散课程(DisCL)”。DisCL为每个训练阶段调整图像合成的图像引导水平:它识别并关注模型的困难样本,并评估合成图像最有效的引导水平以改善困难数据的学习。我们将DisCL应用于两个具有挑战性的任务:长尾(LT)分类和从低质量数据中学习。它专注于高质量的低引导图像以学习原型特征,作为学习可能多样性或质量较弱的高引导图像的预热。大量实验表明,在iWildCam数据集上应用DisCL时,OOD和ID宏平均准确率分别提升了2.7%和2.1%。在ImageNet-LT上,DisCL将基础模型的尾类准确率从4.4%提升至23.64%,并带来全类准确率4.02%的提升。

英文摘要:

Low-quality or scarce data has posed significant challenges for training deep neural networks in practice. While classical data augmentation cannot contribute very different new data, diffusion models opens up a new door to build self-evolving AI by generating high-quality and diverse synthetic data through text-guided prompts. However, text-only guidance cannot control synthetic images' proximity to the original images, resulting in out-of-distribution data detrimental to the model performance. To overcome the limitation, we study image guidance to achieve a spectrum of interpolations between synthetic and real images. With stronger image guidance, the generated images are similar to the training data but hard to learn. While with weaker image guidance, the synthetic images will be easier for model but contribute to a larger distribution gap with the original data. The generated full spectrum of data enables us to build a novel "Diffusion Curriculum (DisCL)". DisCL adjusts the image guidance level of image synthesis for each training stage: It identifies and focuses on hard samples for the model and assesses the most effective guidance level of synthetic images to improve hard data learning. We apply DisCL to two challenging tasks: long-tail (LT) classification and learning from low-quality data. It focuses on lower-guidance images of high-quality to learn prototypical features as a warm-up of learning higher-guidance images that might be weak on diversity or quality. Extensive experiments showcase a gain of 2.7% and 2.1% in OOD and ID macro-accuracy when applying DisCL to iWildCam dataset. On ImageNet-LT, DisCL improves the base model's tail-class accuracy from 4.4% to 23.64% and leads to a 4.02% improvement in all-class accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑