arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.01907cs.CVcs.LG
收藏

ICCV 2023 Perception Test Challenge 2023 解决方案——任务 6——Grounded videoQA

The Solution for the ICCV 2023 Perception Test Challenge 2023 -- Task 6 -- Grounded videoQA

Nanjing University of Science and Technology(南京理工大学)

Hailiang Zhang, Dian Chao, Zhihao Guan, Yang Yang

AI总结:

针对官方视觉定位与跟踪基线在目标缺失帧及时序跟踪类问题上的不足,提出结合 VALOR 作答与 TubeDETR 生成目标边界框的两阶段 Grounded videoQA 方案。

摘要:

本文介绍了一种基于定位的视频问答解决方案。研究表明,视频问答的固定官方基线方法包含两个主要步骤:视觉定位和目标跟踪。然而,在初始步骤中出现了一个重大挑战,即所选帧可能缺少清晰可辨的目标对象。此外,单幅图像无法回答诸如“追踪该人物第一次倾倒时所用的容器”之类的问题。为解决这一问题,我们提出了一种替代的两阶段方法:(1)首先利用 VALOR 模型根据视频信息回答问题;(2)将已回答的问题与其相应答案拼接。最后,采用 TubeDETR 为目标生成边界框。

英文摘要:

In this paper, we introduce a grounded video question-answering solution. Our research reveals that the fixed official baseline method for video question answering involves two main steps: visual grounding and object tracking. However, a significant challenge emerges during the initial step, where selected frames may lack clearly identifiable target objects. Furthermore, single images cannot address questions like "Track the container from which the person pours the first time." To tackle this issue, we propose an alternative two-stage approach:(1) First, we leverage the VALOR model to answer questions based on video information.(2) concatenate the answered questions with their respective answers. Finally, we employ TubeDETR to generate bounding boxes for the targets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑