UGC-VideoCaptioner：一个全模态UGC视频细节描述模型及新基准
UGC-VideoCaptioner: An Omni UGC Video Detail Caption Model and New Benchmarks
AI总结：
针对现有视频描述基准和模型重视觉轻音频、缺乏全模态数据与高效模型的问题，提出UGC-VideoCap全模态UGC视频描述基准及3B参数的UGC-VideoCaptioner模型，通过两阶段训练实现有限数据下的高效适配与高性能表现。
摘要：
现实世界的用户生成视频（尤其是TikTok等平台上的内容）往往包含丰富且交织的音视频内容。然而，现有视频描述基准和模型仍以视觉为核心，忽视了音频在传递场景动态、说话者意图和叙事语境方面的关键作用。全模态数据集的缺失以及轻量且高性能模型的不足，阻碍了细粒度多模态视频理解领域的发展。为应对这些挑战，我们提出了UGC-VideoCap——一个专为短视频形式的用户生成视频的精细化全模态描述设计的全新基准与模型框架。与以往数据集不同，UGC-VideoCap强调音频与视觉模态的平衡融合，包含1000条TikTok视频，通过结构化的三阶段人在环路流水线标注，覆盖纯音频、纯视觉以及音视频联合语义。该基准还包含4000个精心构建的问答对，用于检验单模态与跨模态理解能力。与数据集配套，我们提出了UGC-VideoCaptioner(3B)——一个参数量为3B的描述模型，由Gemini 2.5 Flash蒸馏得到。通过一种新颖的两阶段训练策略（先监督微调，再进行组相对策略优化GRPO），我们的方法能够在有限数据下实现高效适配，同时保持具有竞争力的性能。我们的基准与模型共同为无约束现实UGC场景下的全模态视频描述研究提供了高质量基础与数据高效的解决方案。
英文摘要：
Real-world user-generated videos, especially on platforms like TikTok, often feature rich and intertwined audio visual content. However, existing video captioning benchmarks and models remain predominantly visual centric, overlooking the crucial role of audio in conveying scene dynamics, speaker intent, and narrative context. This lack of omni datasets and lightweight, capable models hampers progress in fine grained, multimodal video understanding. To address these challenges, we introduce UGC-VideoCap, a new benchmark and model framework specifically designed for detailed omnimodal captioning of short form user-generated videos. Unlike prior datasets, UGC-VideoCap emphasizes balanced integration of audio and visual modalities, featuring 1000 TikTok videos annotated through a structured three stage human-in-the-loop pipeline covering audio only, visual only, and joint audio visual semantics. The benchmark also includes 4000 carefully crafted QA pairs probing both unimodal and cross modal understanding. Alongside the dataset, we propose UGC-VideoCaptioner(3B), a 3B parameter captioning model distilled from Gemini 2.5 Flash. Using a novel two-stage training strategy supervised fine tuning followed by Group Relative Policy Optimization (GRPO), our approach enables efficient adaptation from limited data while maintaining competitive performance. Together, our benchmark and model offer a high-quality foundation and a data-efficient solution for advancing omnimodal video captioning in unconstrained real-world UGC settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
