面向深度视频理解的查询感知长视频定位与关系判别
Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding
AI总结：
本文提出一种基于图像-语言预训练模型的查询感知方法，用于长视频定位与关系判别，无需完整电影级知识图谱，在DVU挑战赛中取得优异成绩。
摘要：
视频和社交媒体内容的激增凸显了对多媒体数据进行深度理解的需求。现有成熟的视频理解技术在处理短格式及仅需浅层理解的内容时表现良好，但在处理需要深度理解和推理的长格式视频时表现不佳。深度视频理解（DVU）挑战赛旨在突破多模态提取、融合与分析的边界，以解决整体分析长视频并提取有用知识来解决不同类型查询的问题。本文提出了一种查询感知方法用于长视频定位与关系判别，利用图像-语言预训练模型。该模型能够熟练地选择与查询相关的帧，从而无需完整的电影级知识图谱。我们的方法在两组电影级查询中分别获得了第一名和第四名。充分的实验和最终排名证明了其有效性和鲁棒性。
英文摘要：
The surge in video and social media content underscores the need for a deeper understanding of multimedia data. Most of the existing mature video understanding techniques perform well with short formats and content that requires only shallow understanding, but do not perform well with long format videos that require deep understanding and reasoning. Deep Video Understanding (DVU) Challenge aims to push the boundaries of multimodal extraction, fusion, and analytics to address the problem of holistically analyzing long videos and extract useful knowledge to solve different types of queries. This paper introduces a query-aware method for long video localization and relation discrimination, leveraging an imagelanguage pretrained model. This model adeptly selects frames pertinent to queries, obviating the need for a complete movie-level knowledge graph. Our approach achieved first and fourth positions for two groups of movie-level queries. Sufficient experiments and final rankings demonstrate its effectiveness and robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
