DIBS：通过伪边界丰富与在线优化，利用未标注视频增强密集视频字幕生成
DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinement
AI总结：
提出 DIBS 预训练框架，利用 LLM 生成伪字幕与边界并在线迭代优化，仅用 Vid2Seq 0.4% 的未标注视频即在多个 DVC 基准上取得更优表现。
摘要：
我们提出了 Dive Into the BoundarieS（DIBS），这是一种面向密集视频字幕生成（DVC）的新型预训练框架，致力于提升从未标注视频生成的事件字幕及其关联伪事件边界的质量。通过利用多样化大型语言模型（LLM）的能力，我们生成丰富的、面向 DVC 的字幕候选，并在多个精心设计的目标下优化相应的伪边界，这些目标考虑了多样性、以事件为中心、时间顺序和连贯性。此外，我们进一步提出一种新颖的在线边界优化策略，在训练过程中迭代提升伪边界质量。我们开展了全面实验，以检验所提出技术组件的有效性。通过利用大量未标注视频数据（如 HowTo100M），我们在 YouCook2 和 ActivityNet 等标准 DVC 数据集上取得显著进展。我们在大多数指标上优于此前的最先进方法 Vid2Seq，而实现这一点仅使用了 Vid2Seq 预训练所用未标注视频数据的 0.4%。
英文摘要：
We present Dive Into the BoundarieS (DIBS), a novel pretraining framework for dense video captioning (DVC), that elaborates on improving the quality of the generated event captions and their associated pseudo event boundaries from unlabeled videos. By leveraging the capabilities of diverse large language models (LLMs), we generate rich DVC-oriented caption candidates and optimize the corresponding pseudo boundaries under several meticulously designed objectives, considering diversity, event-centricity, temporal ordering, and coherence. Moreover, we further introduce a novel online boundary refinement strategy that iteratively improves the quality of pseudo boundaries during training. Comprehensive experiments have been conducted to examine the effectiveness of the proposed technique components. By leveraging a substantial amount of unlabeled video data, such as HowTo100M, we achieve a remarkable advancement on standard DVC datasets like YouCook2 and ActivityNet. We outperform the previous state-of-the-art Vid2Seq across a majority of metrics, achieving this with just 0.4% of the unlabeled video data used for pre-training by Vid2Seq.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
