针对官方视觉定位与跟踪基线在目标缺失帧及时序跟踪类问题上的不足,提出结合 VALOR 作答与 TubeDETR 生成目标边界框的两阶段 Grounded videoQA 方案。
摘要:
本文介绍了一种基于定位的视频问答解决方案。研究表明,视频问答的固定官方基线方法包含两个主要步骤:视觉定位和目标跟踪。然而,在初始步骤中出现了一个重大挑战,即所选帧可能缺少清晰可辨的目标对象。此外,单幅图像无法回答诸如“追踪该人物第一次倾倒时所用的容器”之类的问题。为解决这一问题,我们提出了一种替代的两阶段方法:(1)首先利用 VALOR 模型根据视频信息回答问题;(2)将已回答的问题与其相应答案拼接。最后,采用 TubeDETR 为目标生成边界框。
英文摘要:
In this paper, we introduce a grounded video question-answering solution. Our research reveals that the fixed official baseline method for video question answering involves two main steps: visual grounding and object tracking. However, a significant challenge emerges during the initial step, where selected frames may lack clearly identifiable target objects. Furthermore, single images cannot address questions like "Track the container from which the person pours the first time." To tackle this issue, we propose an alternative two-stage approach:(1) First, we leverage the VALOR model to answer questions based on video information.(2) concatenate the answered questions with their respective answers. Finally, we employ TubeDETR to generate bounding boxes for the targets.
MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering
Lanzhou University(兰州大学);National University of Singapore(新加坡国立大学);Sun Yat-sen University(中山大学);Jinan University(暨南大学);Nanyang Technological University(南洋理工大学);Peking University(北京大学);University of Science and Technology of China(中国科学技术大学);Hefei University of Technology(合肥工业大学)
SurgViVQA: Temporally-Grounded Video Question Answering for Surgical Scene Understanding
Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB)(电子、信息与生物工程系);Politecnico di Milano(米兰理工大学);IRCCS Humanitas Research Hospital(IRCCS人类itas研究医院);UCL Hawkes Institute and Department of Computer Science(UCL Hawkes研究所和计算机科学系);University College London(伦敦大学学院);University of Manchester(曼彻斯特大学)
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models
University of California, Davis(加州大学戴维斯分校);Virginia Tech(弗吉尼亚理工大学);The Chinese University of Hong Kong(香港中文大学);NVIDIA;Adobe Research(Adobe研究);Fudan University(复旦大学);Meta AI
ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore(新加坡高性能计算研究所,科技研究局);Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore(科技研究局前沿人工智能研究中心,新加坡);College of Computing and Data Science, Nanyang Technological University, Singapore(新加坡南洋理工大学计算与数据科学学院)
VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotations
University of Science and Technology of China(中国科学技术大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Beihang University(北京航空航天大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University(浙江大学);State Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究所)
Video-EM: Event-Centric Episodic Memory for Long-Form Video Understanding
City University of Hong Kong(香港城市大学);University of Science and Technology of China(中国科学技术大学);Tianjin University(天津大学);Nanjing University(南京大学);Zhejiang University(浙江大学);The Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究所(TeleAI))