合成图像用于模型训练的缩放定律……目前而言
Scaling Laws of Synthetic Images for Model Training ... for Now
AI总结：
本文研究文本生成合成图像训练分类器与 CLIP 的缩放定律，发现提示、引导尺度和生成模型类型影响显著，并明确其有效适用场景。
摘要：
近期，文本到图像模型的重大进展开启了利用合成图像训练视觉系统的可能性，这有望克服大规模收集精选数据的困难。然而，随着越来越多的合成数据被加入训练集，这些模型在规模扩大时的表现尚不清楚。本文研究由最先进文本到图像模型生成的合成图像在训练监督模型时的缩放定律，这些模型包括带标签监督的图像分类器，以及带语言监督的 CLIP。我们识别出若干显著影响缩放行为的因素，包括文本提示、无分类器引导尺度以及文本到图像模型的类型。在对这些因素进行调优后，我们观察到：在 CLIP 训练中，合成图像表现出与真实图像相似的缩放趋势，但效果略逊；而在训练监督式图像分类器时，其缩放表现明显较差。分析表明，造成这种不佳表现的主要原因是现成文本到图像模型无法生成某些概念，这一限制显著损害了图像分类器的训练。我们的发现还表明，在以下场景中扩大合成数据规模可能尤其有效：（1）监督问题的真实图像供应有限（例如 ImageNet 中少于 50 万张图像）；（2）评估数据集与训练数据存在显著差异，即分布外场景；或（3）合成数据与真实图像结合使用，正如 CLIP 模型训练中所展示的那样。
英文摘要：
Recent significant advances in text-to-image models unlock the possibility of training vision systems using synthetic images, potentially overcoming the difficulty of collecting curated data at scale. It is unclear, however, how these models behave at scale, as more synthetic data is added to the training set. In this paper we study the scaling laws of synthetic images generated by state of the art text-to-image models, for the training of supervised models: image classifiers with label supervision, and CLIP with language supervision. We identify several factors, including text prompts, classifier-free guidance scale, and types of text-to-image models, that significantly affect scaling behavior. After tuning these factors, we observe that synthetic images demonstrate a scaling trend similar to, but slightly less effective than, real images in CLIP training, while they significantly underperform in scaling when training supervised image classifiers. Our analysis indicates that the main reason for this underperformance is the inability of off-the-shelf text-to-image models to generate certain concepts, a limitation that significantly impairs the training of image classifiers. Our findings also suggest that scaling synthetic data can be particularly effective in scenarios such as: (1) when there is a limited supply of real images for a supervised problem (e.g., fewer than 0.5 million images in ImageNet), (2) when the evaluation dataset diverges significantly from the training data, indicating the out-of-distribution scenario, or (3) when synthetic data is used in conjunction with real images, as demonstrated in the training of CLIP models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
