Summer-22B：视频基础模型大规模数据集工程与训练的系统方法
Summer-22B: A Systematic Approach to Dataset Engineering and Training at Scale for Video Foundation Model
AI总结：
Summer-22B通过系统化数据集工程与训练方法，实现了视频基础模型的规模化训练，重点在于数据集构建、参数优化及架构设计。
摘要：
我们描述了训练Summer-22B视频基础模型的过程，该模型是从零开始开发的。本报告记录了从原始视频采集到训练约5000万片段的模型的过程，涵盖了工程挑战、设计决策和经验教训。我们结合了元数据驱动的数据集整理、多阶段过滤、μP参数化和超球面约束优化的方法。我们开发了Lavender数据系统用于数据集管理，并采用了推理感知的架构选择。我们分享了在我们设置中有效的观察：数据集工程消耗了大部分努力，架构变体的差异比我们预期的小，而μP超参数转移在几何约束下也表现有效。我们希望这份报告能对进行类似项目的其他人有所帮助。
英文摘要：
We describe our experience training Summer-22B, a video foundation model developed from scratch. This report documents the engineering challenges, design decisions, and lessons learned while scaling from raw footage collection to a functional model trained on approximately 50 million clips. We outline our approach combining metadata-driven dataset curation, multi-stage filtering, $μ$P parameterization, and hypersphere-constrained optimization. We developed the Lavender Data system for dataset management and adopted inference-aware architectural choices. We share observations on what worked in our setting: dataset engineering consumed the majority of effort, architectural variants showed smaller differences than we expected, and $μ$P hyperparameter transfer appeared effective even under geometric constraints. We hope this account proves useful to others undertaking similar projects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
