ROVI：用于开放词汇实例级文本到图像生成的VLM-LLM重新标注数据集
ROVI: A VLM-LLM Re-Captioned Dataset for Open-Vocabulary Instance-Grounded Text-to-Image Generation
AI总结：
ROVI通过VLM-LLM重新标注策略生成高质量合成数据集，提升开放词汇实例级文本到图像生成的定位精度与提示保真度。
摘要：
我们提出了ROVI，一个用于实例级文本到图像生成的高质量合成数据集，通过标注100万张精选网络图像构建。我们的关键创新是一种称为“重新标注”的策略，聚焦于预检测阶段，其中视觉语言模型（VLM）生成全面的视觉描述，随后由大语言模型（LLM）处理，提取出供开放词汇检测器（OVDs）检测的潜在类别扁平列表。这种方法产生了一个与实例标注固有关联的全局提示，同时捕获了人类通常忽略的次要视觉元素。评估表明，ROVI在图像质量和分辨率上超越了现有的检测数据集，同时包含两个数量级更多的类别，并具有开放词汇特性。为演示目的，在ROVI上训练的文本到图像模型GLIGEN在实例定位准确性、提示保真度和美学质量方面显著优于最先进的替代方案。我们的数据集和可复现流程可在https://github.com/CihangPeng/ROVI获取。
英文摘要：
We present ROVI, a high-quality synthetic dataset for instance-grounded text-to-image generation, created by labeling 1M curated web images. Our key innovation is a strategy called re-captioning, focusing on the pre-detection stage, where a VLM (Vision-Language Model) generates comprehensive visual descriptions that are then processed by an LLM (Large Language Model) to extract a flat list of potential categories for OVDs (Open-Vocabulary Detectors) to detect. This approach yields a global prompt inherently linked to instance annotations while capturing secondary visual elements humans typically overlook. Evaluations show that ROVI exceeds existing detection datasets in image quality and resolution while containing two orders of magnitude more categories with an open-vocabulary nature. For demonstrative purposes, a text-to-image model GLIGEN trained on ROVI significantly outperforms state-of-the-art alternatives in instance grounding accuracy, prompt fidelity, and aesthetic quality. Our dataset and reproducible pipeline are available at https://github.com/CihangPeng/ROVI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
