arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01370cs.CVcs.AIcs.LG
收藏

PolyGen: 通过多生成器集合实现完全合成的视觉-语言训练

PolyGen: Fully Synthetic Vision-Language Training via Multi-Generator Ensembles

Politecnico di Milano, AIRLab(米兰理工学院,AIRLab);University of Tokyo, Computer Vision and Media Lab(东京大学,计算机视觉与媒体实验室)

Leonardo Brusini, Cristian Sbrolli, Eugenio Lomurno, Toshihiko Yamasaki, Matteo Matteucci

AI总结:

PolyGen通过多生成器集合提升合成数据的视觉-语言预训练效果,实现更稳健的特征空间和更高的多任务性能。

摘要:

合成数据为视觉-语言预训练提供了可扩展的解决方案,但当前最先进的方法通常依赖于扩大单一生成器的规模,这引入了生成器特定的频谱偏见并限制了特征多样性。在本工作中,我们介绍了PolyGen框架,该框架通过优先考虑流形覆盖和组合严谨性而非简单的数据集大小来重新定义合成数据的构建。PolyGen采用多石方法在架构上不同的生成器的交集上进行训练,有效地消除了模型特定的 artifacts。此外,我们引入了程序化困难负样本课程,以强制细粒度的语法理解。通过结构性地重新分配相同的数据预算从独特的描述到多源变体,PolyGen实现了更稳健的特征空间,在聚合多任务基准和SugarCrepe++组合性基准上分别优于领先的单源基线(SynthCLIP)+19.0%和+9.1%。这些结果表明,结构多样性比单纯增加单源样本的体积是一种更数据高效的扩展定律。

英文摘要:

Synthetic data offers a scalable solution for vision-language pre-training, yet current state-of-the-art methods typically rely on scaling up a single generative backbone, which introduces generator-specific spectral biases and limits feature diversity. In this work, we introduce PolyGen, a framework that redefines synthetic data construction by prioritizing manifold coverage and compositional rigor over simple dataset size. PolyGen employs a Polylithic approach to train on the intersection of architecturally distinct generators, effectively marginalizing out model-specific artifacts. Additionally, we introduce a Programmatic Hard Negative curriculum that enforces fine-grained syntactic understanding. By structurally reallocating the same data budget from unique captions to multi-source variations, PolyGen achieves a more robust feature space, outperforming the leading single-source baseline (SynthCLIP) by +19.0% on aggregate multi-task benchmarks and on the SugarCrepe++ compositionality benchmark (+9.1%). These results demonstrate that structural diversity is a more data-efficient scaling law than simply increasing the volume of single-source samples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑