arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.06355cs.CVcs.AIcs.CL
收藏

基于属性增强注意网络学习的视频问答

Video Question Answering via Attribute-Augmented Attention Network Learning

Zhejiang University(浙江大学)

Yunan Ye, Zhou Zhao, Yimeng Li, Long Chen, Jun Xiao, Yueting Zhuang

摘要:

视频问答是视觉信息检索中的一个具有挑战性的问题,它根据问题对所引用的视频内容提供答案。然而,现有的视觉问答方法主要解决静态图像问题,这可能是无效的视频问答回答由于建模的视频内容的时间动态的不足。本文采用帧级注意力机制对视频问答进行时间动态建模,研究视频问答问题。我们提出的属性增强注意力网络学习框架,使联合帧级属性检测和统一的视频表示学习的视频问答。然后,我们将多步推理过程中,我们提出的注意力网络,以进一步提高性能。我们构建了一个大规模的视频问答数据集。我们在多项选择题和开放式视频问答任务上进行了实验,以证明所提出的方法的有效性。

英文摘要:

Video Question Answering is a challenging problem in visual information retrieval, which provides the answer to the referenced video content according to the question. However, the existing visual question answering approaches mainly tackle the problem of static image question, which may be ineffectively for video question answering due to the insufficiency of modeling the temporal dynamics of video contents. In this paper, we study the problem of video question answering by modeling its temporal dynamics with frame-level attention mechanism. We propose the attribute-augmented attention network learning framework that enables the joint frame-level attribute detection and unified video representation learning for video question answering. We then incorporate the multi-step reasoning process for our proposed attention network to further improve the performance. We construct a large-scale video question answering dataset. We conduct the experiments on both multiple-choice and open-ended video question answering tasks to show the effectiveness of the proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑