BabyVLM-V2：迈向基于发展基础的视觉基础模型预训练与基准测试
BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models
AI总结：
BabyVLM-V2通过纵向多维预训练集、灵活模型和DevCV工具箱，提升婴儿启发式视觉语言模型性能，实验证明其在基准测试中表现优异。
摘要：
早期儿童的发展轨迹为高效预训练视觉基础模型提供了自然目标。我们介绍了BabyVLM-V2，一个以婴儿启发式视觉语言建模为核心的框架，通过纵向、多方面的预训练集、灵活模型以及最重要的DevCV工具箱，显著改进了BabyVLM-V1。预训练集最大化覆盖范围同时最小化对纵向、以婴儿为中心的音频视觉语料库的编辑，产生视频-陈述、图像-陈述和多轮对话数据，模拟婴儿经验。DevCV工具箱将最近发布的NIH Baby Toolbox中的所有视觉相关测量转化为十个多模态任务的基准测试集，涵盖空间推理、记忆和词汇理解，与早期儿童能力对齐。实验结果表明，从头预训练的紧凑模型在DevCV工具箱中表现竞争，某些任务上优于GPT-4o。我们希望这个原则性、统一的BabyVLM-V2框架能加速研究在发展合理预训练视觉基础模型方面的研究。
英文摘要：
Early children's developmental trajectories set up a natural goal for sample-efficient pretraining of vision foundation models. We introduce BabyVLM-V2, a developmentally grounded framework for infant-inspired vision-language modeling that extensively improves upon BabyVLM-V1 through a longitudinal, multifaceted pretraining set, a versatile model, and, most importantly, DevCV Toolbox for cognitive evaluation. The pretraining set maximizes coverage while minimizing curation of a longitudinal, infant-centric audiovisual corpus, yielding video-utterance, image-utterance, and multi-turn conversational data that mirror infant experiences. DevCV Toolbox adapts all vision-related measures of the recently released NIH Baby Toolbox into a benchmark suite of ten multimodal tasks, covering spatial reasoning, memory, and vocabulary understanding aligned with early children's capabilities. Experimental results show that a compact model pretrained from scratch can achieve competitive performance on DevCV Toolbox, outperforming GPT-4o on some tasks. We hope the principled, unified BabyVLM-V2 framework will accelerate research in developmentally plausible pretraining of vision foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
