VisionFoundry: 通过合成图像教授VLMs的视觉感知
VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images
AI总结：
本文提出VisionFoundry，通过任务关键词生成合成数据，提升VLMs在空间理解和视角识别等视觉感知任务上的性能，构建了包含10k图像-问题-答案三元组的VQA数据集，并在多个基准测试中取得显著提升。
摘要：
视觉语言模型（VLMs）在空间理解和视角识别等视觉感知任务上仍存在困难。自然图像数据集对低级视觉技能的监督有限，这促使我们提出一个实践问题：能否通过仅需任务关键词（如深度顺序）生成的合成监督来解决这些问题？为此，我们引入VisionFoundry，一种任务感知的合成数据生成流水线，仅需任务名称作为输入，利用大语言模型（LLMs）生成问题、答案和文本到图像（T2I）提示，然后通过T2I模型合成图像，并与专有VLM验证一致性，无需参考图像或人工标注。使用VisionFoundry，我们构建了包含10k图像-问题-答案三元组的VisionFoundry-10K合成视觉问答（VQA）数据集，覆盖10种任务。在VisionFoundry-10K上训练的模型在视觉感知基准测试中取得显著提升：在MMVP上提升7%，在CV-Bench-3D上提升10%，同时保持更广泛的能力并显示出数据规模增加时的有利扩展行为。我们的结果表明，有限的任务定向监督是这一瓶颈的重要贡献因素，合成监督是朝着更系统训练VLMs的有希望路径。
英文摘要：
Vision-language models (VLMs) still struggle with visual perception tasks such as spatial understanding and viewpoint recognition, largely because natural image datasets provide limited supervision for low-level visual skills. Can targeted synthetic supervision address these weaknesses without reference images or manual annotation? To investigate this, we introduce VisionFoundry, an automated pipeline that takes only a task name as input, uses LLMs to synthesize paired questions, answers, and text-to-image (T2I) prompts, generates images with T2I models, and filters samples via multimodal verification. With VisionFoundry, we construct VisionFoundry-10k, a synthetic VQA dataset spanning 10 perception tasks. Finetuning on VisionFoundry-10k consistently improves perception benchmarks across three open-source backbones (e.g., +6.7% on MMVP-pair and +10.5% on CV-Bench-3D for Qwen2.5-VL-3B-Instruct) while preserving broader capabilities and showing positive data scaling. The same synthetic supervision also yields consistent gains under reinforcement learning (RL) across all three backbones, and the framework remains effective under open-source synthesis and self-verification. Our findings demonstrate that automated synthetic supervision offers an effective and scalable path toward systematic VLM training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
