知道聚焦何处：用于视频定位的事件感知 Transformer
Knowing Where to Focus: Event-aware Transformer for Video Grounding
AI总结：
针对基于 DETR 的视频定位中输入无关查询缺乏时间位置信息的问题，提出事件感知动态时刻查询，通过 slot attention 事件推理和门控融合 transformer 时刻推理提升性能。
摘要：
近期基于 DETR 的视频定位模型通过学习时刻查询，使模型能够直接预测时刻时间戳，而无需任何手工组件，例如预定义候选框或非极大值抑制。然而，它们与输入无关的时刻查询不可避免地忽视了视频固有的时间结构，只能提供有限的位置信息。本文构建了一种事件感知的动态时刻查询，使模型能够考虑视频中随输入变化的内容和位置信息。为此，我们提出两个层次的推理：1）事件推理，利用 slot attention 机制捕捉构成给定视频的显著事件单元；2）时刻推理，通过门控融合 transformer 层将时刻查询与给定句子融合，并学习时刻查询与视频-句子表示之间的交互，以预测时刻时间戳。大量实验证明了事件感知动态时刻查询的有效性和效率，在多个视频定位基准上优于最先进方法。
英文摘要：
Recent DETR-based video grounding models have made the model directly predict moment timestamps without any hand-crafted components, such as a pre-defined proposal or non-maximum suppression, by learning moment queries. However, their input-agnostic moment queries inevitably overlook an intrinsic temporal structure of a video, providing limited positional information. In this paper, we formulate an event-aware dynamic moment query to enable the model to take the input-specific content and positional information of the video into account. To this end, we present two levels of reasoning: 1) Event reasoning that captures distinctive event units constituting a given video using a slot attention mechanism; and 2) moment reasoning that fuses the moment queries with a given sentence through a gated fusion transformer layer and learns interactions between the moment queries and video-sentence representations to predict moment timestamps. Extensive experiments demonstrate the effectiveness and efficiency of the event-aware dynamic moment queries, outperforming state-of-the-art approaches on several video grounding benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
