TEMPURA：面向动作推理的时间事件掩码预测与理解
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action
AI总结：
TEMPURA提出两阶段训练框架，通过掩码事件预测和视频分割密集字幕生成，结合事件推理与时间分割，显著提升视觉语言模型的视频时间理解能力。
摘要：
理解视频中的因果事件关系并实现细粒度的时间定位，对于视觉语言模型（VLMs）而言仍具挑战性。我们提出TEMPURA（面向动作推理的时间事件掩码预测与理解），一个两阶段训练框架，旨在增强VLM的视频时间理解能力。受语言建模中填充技术的启发，TEMPURA首先执行掩码事件预测，学习从密集事件标注中重建缺失事件并生成逐步因果解释。随后，它学习视频分割和密集字幕生成，将视频分解为带有详细时间戳对齐描述的非重叠事件。我们在VER上训练TEMPURA，这是一个包含50万视频的大规模数据集，标注有时间对齐的事件描述和结构化推理步骤。在视频时间定位和精彩片段检测基准上的实验表明，TEMPURA显著提升了不同模型家族和规模下的强基础VLM，证实了将事件级推理与细粒度时间分割相结合是视频时间理解的有效方案。
英文摘要：
Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLMs). We propose TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action), a two-stage training framework that enhances the video temporal understanding of VLMs. Inspired by infilling techniques in language modeling, TEMPURA first performs masked event prediction, learning to reconstruct missing events and generate step-by-step causal explanations from dense event annotations. It then learns video segmentation and dense captioning, decomposing videos into non-overlapping events with detailed, timestamp-aligned descriptions. We train TEMPURA on VER, our large-scale dataset of 500K videos annotated with temporally aligned event descriptions and structured reasoning steps. Experiments on video temporal grounding and highlight detection benchmarks show that TEMPURA substantially improves strong base VLMs across model families and scales, confirming that combining event-level reasoning with fine-grained temporal segmentation is an effective recipe for video temporal understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
