Auto Cherry-Picker：由语言驱动的高质量生成数据学习
Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Language
AI总结：
针对扩散模型生成数据用于下游感知任务的分布对齐、规模与质量难题，提出ACP框架，借助LLM生成布局描述、可控扩散模型出图、CLIS指标筛选，可规模化生成高质量跨模态样本，显著提升下游模型性能。
摘要：
扩散模型能够生成逼真且多样化的图像，有望为数据密集型感知任务缓解数据可用性问题。然而，利用这些模型通过合成数据提升下游任务性能存在若干挑战，包括与真实数据分布对齐、扩大合成样本规模以及保障数据质量。为填补这些缺口，我们提出了Auto Cherry-Picker（ACP，自动精选器），这是一种新型框架，可大规模生成高质量跨模态训练样本，用于增强感知与多模态训练。ACP首先利用大语言模型（LLMs），基于真实数据先验中的目标组合采样描述文本与布局，无需真实图像的字幕或标注。接着，我们使用现成的可控扩散模型生成多张图像。随后，通过一套全面设计的指标——复合布局与图像分数（CLIS）对生成数据进行筛选优化，以保障质量。我们定制的合成高质量样本在多种场景下均能提升性能，尤其在应对长尾分布与数据集不平衡相关的挑战时效果显著。下游任务的实验结果表明，ACP可显著提升现有模型的性能。此外，我们发现CLIS与下游任务的性能提升之间存在正相关关系。这一发现展现了评估指标在各类视觉感知与多模态大语言模型（MLLM）任务中的应用潜力。
英文摘要：
Diffusion models can generate realistic and diverse images, potentially facilitating data availability for data-intensive perception tasks. However, leveraging these models to boost performance on downstream tasks with synthetic data poses several challenges, including aligning with real data distribution, scaling synthetic sample volumes, and ensuring their quality. To bridge these gaps, we present \textbf{A}uto \textbf{C}herry-\textbf{P}icker (ACP), a novel framework that generates high-quality cross-modality training samples at scale to augment perception and multi-modal training. ACP first uses LLMs to sample descriptions and layouts based on object combinations from real data priors, eliminating the need for ground truth image captions or annotations. Next, we use an off-the-shelf controllable diffusion model to generate multiple images. Then, the generated data are refined using a comprehensively designed metric, Composite Layout and Image Score (CLIS), to ensure quality. Our customized synthetic high-quality samples boost performance in various scenarios, especially in addressing challenges associated with long-tailed distribution and imbalanced datasets. Experiment results on downstream tasks demonstrate that ACP can significantly improve the performance of existing models. In addition, we find a positive correlation between CLIS and performance gains in downstream tasks. This finding shows the potential for evaluation metrics as the role for various visual perception and MLLM tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
