arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.00136cs.CVcs.AIcs.LG
收藏

面向视频知识图谱提取的检测-融合方法

Detection-Fusion for Knowledge Graph Extraction from Videos

University of Oxford(牛津大学);University of Edinburgh(爱丁堡大学);Vienna University of Technology(维也纳工业大学)

Taniya Das, Louis Mahon, Thomas Lukasiewicz

AI总结:

本文提出一种基于深度学习的检测-融合方法,通过先预测个体对再预测关系来从视频中构建知识图谱,并引入背景知识扩展,以替代自然语言描述并克服其难以处理和评估的缺陷。

摘要:

视频理解领域中的一项具有挑战性的任务是从视频输入中提取语义内容。现有系统大多使用语言模型以自然语言句子描述视频,但这存在若干重大缺陷。此类系统可能过度依赖语言模型组件,并基于自然语言文本中的统计规律而非视频的视觉内容来生成输出。此外,自然语言标注难以被计算机直接处理,难以用性能指标进行评估,也难以翻译成另一种自然语言。本文提出一种用知识图谱对视频进行标注的方法,从而避免上述问题。具体而言,我们提出一个基于深度学习的模型来完成该任务,该模型首先预测个体对,然后预测它们之间的关系。此外,我们还提出了该模型的一个扩展,用于在知识图谱构建过程中引入背景知识。

英文摘要:

One of the challenging tasks in the field of video understanding is extracting semantic content from video inputs. Most existing systems use language models to describe videos in natural language sentences, but this has several major shortcomings. Such systems can rely too heavily on the language model component and base their output on statistical regularities in natural language text rather than on the visual contents of the video. Additionally, natural language annotations cannot be readily processed by a computer, are difficult to evaluate with performance metrics and cannot be easily translated into a different natural language. In this paper, we propose a method to annotate videos with knowledge graphs, and so avoid these problems. Specifically, we propose a deep-learning-based model for this task that first predicts pairs of individuals and then the relations between them. Additionally, we propose an extension of our model for the inclusion of background knowledge in the construction of knowledge graphs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑