流式视频中的时序句子定位
Temporal Sentence Grounding in Streaming Videos
AI总结：
针对流式视频需在线处理且无未来帧的问题，提出TwinNet与语言引导特征压缩器，实现TSGSV并在多个数据集上验证优越性能。
摘要：
本文旨在解决一项新颖任务——流式视频中的时序句子定位（Temporal Sentence Grounding in Streaming Videos，TSGSV）。TSGSV的目标是评估视频流与给定句子查询之间的相关性。与常规视频不同，流式视频从特定来源持续获取，并且在监控和直播分析等许多应用中，总需要对其进行即时处理。因此，TSGSV具有挑战性，因为它要求模型在没有未来帧的情况下进行推理，并有效处理较长的历史帧，而早期方法尚未涉及这些问题。为专门应对上述挑战，我们提出两种新颖方法：（1）TwinNet结构，使模型能够了解即将发生的事件；（2）语言引导的特征压缩器，用于消除冗余视觉帧，并增强与查询相关的帧。我们使用ActivityNet Captions、TACoS和MAD数据集进行了大量实验。结果证明了所提方法的优越性。系统的消融研究也证实了它们的有效性。
英文摘要：
This paper aims to tackle a novel task - Temporal Sentence Grounding in Streaming Videos (TSGSV). The goal of TSGSV is to evaluate the relevance between a video stream and a given sentence query. Unlike regular videos, streaming videos are acquired continuously from a particular source, and are always desired to be processed on-the-fly in many applications such as surveillance and live-stream analysis. Thus, TSGSV is challenging since it requires the model to infer without future frames and process long historical frames effectively, which is untouched in the early methods. To specifically address the above challenges, we propose two novel methods: (1) a TwinNet structure that enables the model to learn about upcoming events; and (2) a language-guided feature compressor that eliminates redundant visual frames and reinforces the frames that are relevant to the query. We conduct extensive experiments using ActivityNet Captions, TACoS, and MAD datasets. The results demonstrate the superiority of our proposed methods. A systematic ablation study also confirms their effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
