arXiv 2501.148182025-01-20cs.CVcs.AIcs.LG

Eagle 2：从零构建面向前沿视觉语言模型的后训练数据策略

Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models

Nanjing University（南京大学）；Tsinghua University（清华大学）；Hong Kong Polytechnic University（香港理工大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；New York University（纽约大学）；NVIDIA（英伟达）

Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, Nadine Chang, Karan Sapra, Ama… 展开作者

Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, Nadine Chang, Karan Sapra, Amala Sanjay Deshmukh, Tuomas Rintamaki, Matthieu Le, Ilia Karmanov, Lukas Voegtle, Philipp Fischer, De-An Huang, Timo Roman, Tong Lu, Jose M. Alvarez, Bryan Catanzaro, Jan Kautz, Andrew Tao, Guilin Liu, Zhiding Yu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文从以数据为中心的视角探讨视觉语言模型的后训练，从零构建了后训练数据策略、训练配方与模型设计，推出了高性能模型家族 Eagle2，其中 Eagle2-9B 在多模态基准上达到 SOTA，媲美 70B 参数的竞争模型。

摘要：

近期，开源视觉语言模型（VLMs）在将其能力提升至接近专有前沿模型方面取得了可喜进展。然而，大多数开源模型仅发布最终模型权重，导致数据策略和实现的关键细节在很大程度上不透明。在这项工作中，我们从以数据为中心的视角探讨 VLM 后训练，展示了数据策略在开发前沿 VLM 中的关键作用。通过从零开始研究和构建我们的后训练数据策略，我们分享了开发过程的详细见解，旨在造福开源社区开发具有竞争力的模型。我们引入的数据策略，连同训练配方和模型设计，催生了一系列名为 Eagle2 的高性能 VLM。具体而言，Eagle2-9B 在各种多模态基准测试中取得了最先进的结果，匹敌了某些参数量高达 70B 的竞争模型。

英文摘要：

Recently, promising progress has been made by open-source vision-language models (VLMs) in bringing their capabilities closer to those of proprietary frontier models. However, most open-source models only publish their final model weights, leaving the critical details of data strategies and implementation largely opaque. In this work, we address VLM post-training from a data-centric perspective, showing the key role of data strategy in developing frontier VLMs. By studying and building our post-training data strategy from scratch, we share detailed insights into the development processes, aiming to benefit the development of competitive models for the open-source community. Our introduced data strategy, together with training recipes and model design, leads to a family of performant VLMs named Eagle2. Specifically, Eagle2-9B achieves state-of-the-art results across various multimodal benchmarks, matching certain competitive models with up to 70B parameters.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。